Deep water monitoring-oriented cloud edge collaborative large model learnable quantification method and system

Abstract

The invention discloses a deep water monitoring-oriented cloud edge collaborative large model learnable quantification method and system. The method is realized through a cloud edge collaboration framework: an edge monitoring device processes a simple task locally by using a lightweight model, and triggers a cloud collaboration process for a complex task. And at the cloud end, the selected large base model is customized and quantized through LQGDC. The LQGDC algorithm performs model quantization through parameter optimization guided by learnable transformation and distribution correction. Therefore, information is reserved to the maximum extent, it is ensured that the output of the quantization block is aligned with the full-precision model in the numerical and semantic directions, more importantly, an error correction item is introduced, quantization errors accumulated layer by layer are actively compensated and corrected, and the limitation of error chain type transmission is broken through. The optimal lightweight model after customization quantification and performance verification can be safely distributed to the edge device initiating the request, and is used for efficiently executing off-line reasoning tasks such as water environment risk pre-judgment and multi-source data fusion verification.

Description

Technical Field

[0001] This invention relates to a neural network model compression technology applied to the field of water environment monitoring and analysis in large reservoirs. Specifically, it relates to a learnable quantization method and system for large models (LM) based on cloud-edge collaboration, belonging to the interdisciplinary field of large artificial intelligence model compression, edge computing, and air-water collaborative monitoring technology. Background Technology

[0002] With the rapid development of large-scale models, the number of model parameters has expanded from billions to hundreds of billions; for example, GPT-3 has 175 billion parameters, and the LLaMA series covers 7 billion to 65 billion parameters. These large-scale pre-trained models have demonstrated outstanding capabilities in tasks such as natural language processing, dialogue generation, code writing, knowledge question answering, and data analysis. Their powerful potential for multi-source data analysis, complex reasoning, and risk early warning is particularly suitable for water environment safety monitoring and ecological protection work in large reservoirs.

[0003] Traditional water environment monitoring and analysis systems mainly rely on physical models and simple statistical analysis, which are insufficient to handle in-depth analysis needs such as complex carbon-nitrogen coupled cycles, spatiotemporal coupling of multiple physical processes, and risk factor identification. Therefore, introducing large-scale models with multi-source data fusion, validation, and decision support capabilities into water environment monitoring systems has become an important technological trend for improving monitoring, early warning, and sustainable management capabilities.

[0004] Currently, water environment monitoring is transforming towards air-water synergy and intelligent analysis. There is an urgent need for efficient fusion and analysis of multi-source, heterogeneous, and multi-scale environmental monitoring data, such as buoy automatic sensing and space-based telemetry hyperspectral data, to achieve real-time intelligent applications like algal bloom early warning and pollution source tracing. This requires efficient analysis of multi-source, heterogeneous, and multi-scale environmental data collected from devices such as 200m deep-water buoys and UAV hyperspectral systems to support reservoir ecological security management and sustainable development. However, the enormous computational and storage requirements of these large models pose significant challenges to practical deployment, especially on resource-constrained edge devices (EDs). These devices typically have limited computing power, small memory capacity, and low power budgets, making them unable to support complete large model inference and severely restricting the widespread adoption of large models in edge computing scenarios.

[0005] Deploying large-scale models on edge monitoring devices (EDs) with extremely limited computing and storage resources, such as deep-water buoy stations and unmanned aerial vehicle (UAV) systems, to achieve real-time, offline, and intelligent data analysis and early warning faces the following core technical challenges: The conflict between quantification accuracy and the dynamism of aquatic environmental data: Large reservoirs exhibit significant environmental stratification, including variations in water depth, temperature, and dissolved oxygen. Furthermore, the spatiotemporal distribution of their hydrochemical, aquatic ecological, and sedimentary chemical characteristics varies dramatically, resulting in complex and dynamically changing input data for large models. Existing static quantification methods cannot dynamically adapt to this multi-source, heterogeneous environmental data distribution. When processing key indicators such as permanganate index, chlorophyll fluorescence, and dissolved CO2 concentration, they are prone to severe loss of quantification accuracy, directly impacting the accuracy of algal bloom emergency prediction and pollution source tracing models. For example, current mainstream model quantification methods, such as GPTQ and OmniQuant, while reducing model size to some extent, primarily employ static quantification strategies with fixed quantification parameters, failing to adapt to the diverse data distribution characteristics at different monitoring points. When analyzing medium- to long-term trends in environmental data, large models need to process long-term data, leading to significant memory consumption in the key-value (KV) cache of attention mechanisms. Existing KV caching quantization methods are ineffective at handling dynamic outliers between channels and terms, limiting the effective utilization of long-term environmental data by the models.

[0006] Within the cloud-edge collaborative framework, existing solutions face multiple technical challenges. First, privacy and security issues are prominent; directly uploading monitoring data to the cloud poses a risk of privacy breaches, while fully localized processing is limited by device computing power. Second, communication efficiency is low; frequent model transmissions and large data volumes lead to poor performance in unstable network environments. Third, personalized adaptation capabilities are insufficient; existing quantization methods lack dynamic adaptation mechanisms to the data distribution characteristics of specific devices, resulting in inconsistent performance of the quantized model across different edge devices. Furthermore, different monitoring points (buoy stations, drones) and different monitoring tasks require customized quantization models, but existing solutions lack efficient, low-leakage cloud-edge collaborative customization mechanisms. These issues collectively restrict the effective deployment and application of large-scale models in edge computing environments.

[0007] In summary, the effective deployment of large models in edge computing environments still faces constraints in many aspects, such as quantization accuracy, data dynamism, privacy and security, communication efficiency, and personalized adaptation. Further breakthroughs are urgently needed in quantization methods and cloud-edge collaborative architecture. Summary of the Invention

[0008] Purpose of the invention: This invention provides a learnable quantization method and system for cloud-edge collaborative large-scale models for deep-water monitoring. It overcomes the problems of accuracy loss, high memory consumption of KV cache, and lack of personalized adaptive capabilities of traditional quantization methods on reservoir monitoring edge devices. It achieves dynamic adaptability of quantization parameters to edge data distribution, optimizes KV cache quantization to reduce memory consumption of long text processing, realizes cloud-edge collaboration while protecting privacy, and ensures smooth task execution in scenarios with unstable communication networks.

[0009] Technical Solution: A cloud-edge collaborative large-scale model learnable quantization method for deep-water monitoring. For simple tasks, edge devices utilize locally deployed small models to perform the task. For complex tasks, edge devices upload task data to a cloud server as a calibration dataset. The cloud server determines the base model based on the task. The calibration dataset is used as input to the base model, and a quantization algorithm is executed on the cloud server, outputting multiple quantized models. These quantized models are evaluated on the task, and the optimal model is selected. Finally, the quantized model is distributed to the corresponding edge devices for local inference. The method includes the following steps: Step 1) The edge device assesses the complexity of the local task. When the task complexity is not higher than the preset threshold (e.g., whether it is an emergency warning), the locally deployed lightweight model is called for processing, which is sufficient to meet simple daily needs; when the task complexity is higher than the preset threshold, the cloud-based collaborative quantification process is triggered.

[0010] Step 2) The edge device samples and de-identifies local data to generate a small, representative calibration dataset, which is then uploaded to the cloud server along with device hardware information and task requirements. The cloud server selects a suitable large-scale model (such as LLaMA-13B or Qwen) from the cloud's model pool based on the task requirements and determines the model quantization configuration.

[0011] Step 3) The cloud server executes a distribution-corrected guided learnable quantization (LQGDC) algorithm. This algorithm introduces learnable equivalent transformations: learnable parameters (such as smoothing factors, pruning thresholds, scaling and translation factors) are introduced into the weights, activation values, and key-value buffers of the base model to dynamically adjust the numerical distribution of weights, activation values, and key-value buffers, adapting to the dynamically changing distribution of reservoir monitoring data and making it more suitable for low-bit quantization. Simultaneously, distribution-corrected guided parameter optimization is used to construct a composite loss function that combines mean squared error, cosine similarity, and KL divergence, optimizing the aforementioned learnable parameters block by block. This ensures that the quantized model output approximates the full-precision model to the greatest extent possible in terms of numerical, directional, and attention distributions.

[0012] Step 4) The cloud server performs performance verification on the corrected quantization model and distributes the optimized best quantization model to the corresponding edge devices. The edge devices load the model and efficiently perform local inference tasks such as water environment risk prediction and data fusion verification.

[0013] In step 1), edge devices refer to computationally limited devices such as deep-water buoy control systems, UAV-borne hyperspectral processing units, or shore-based automatic monitoring stations. Complex tasks include, for example, early warning of sudden algal blooms based on air-water collaborative monitoring data, real-time source tracing analysis of pollutant migration paths in reservoir areas, or long-term ecological and environmental trend prediction based on multi-factor coupling. Edge devices distribute simple tasks to smaller models based on task complexity, significantly reducing reliance on cloud services and network communication overhead. The task evaluation module can determine complexity using indicators such as task type, input data length (long time series), or required computation. When a complex task is identified that a local lightweight model cannot handle, the cloud-based collaborative quantification process is triggered, executing step 2).

[0014] In step 2), after the cloud-based collaborative process is triggered: the data desensitization and sampling module on the edge device randomly samples a small portion of data from local data related to the current complex task. This module also randomly samples and desensitizes local monitoring data related to the current complex monitoring task (such as real-time water quality profile data from buoy stations and hyperspectral images from drones) to form a calibration dataset. This dataset, along with device hardware configuration information (such as memory size and supported computational precision) and task requirements (such as "algal bloom early warning" and "pollution source tracing"), is sent to the cloud server. The cloud server selects a suitable large-scale model from the model pool in the cloud based on the task requirements. The model pool pre-stores various sizes and types of open-source or self-developed large models (such as LLaMA-13B, Qwen-7B, etc.), determines the model quantization configuration, and executes the distribution-correction-guided learnable quantization algorithm in step 3.

[0015] In step 3), the cloud server utilizes the calibration dataset uploaded by the edge device to execute an innovative distributed correction-guided learnable quantization (LQGDC) algorithm on the selected pedestal large model. The goal of this algorithm is to generate a low-bit quantization model that highly matches the behavior of the original full-precision model. The execution flow of this step is as follows: LQGDC first initializes and establishes optimization criteria. Before starting quantization optimization, the cloud server first needs to establish an accurate reference baseline for the subsequent optimization process. The cloud server loads the pedestal large model selected in step 2) and the received calibration dataset. Then, the entire calibration dataset is input into the unmodified, full-precision pedestal large model, performing a complete forward propagation. During this process, the cloud server caches and records the following key information: Block-by-block full-precision output ( For each Transformer block in the model, record its full-precision output tensor after receiving full-precision input. This represents the output of that block under ideal conditions.

[0016] Block-by-block full-precision attention output ( ): Records the attention score matrix for each attention head in the model. This reflects the distribution of association weights between different lexical units when the model processes calibration data.

[0017] After initialization, the LQGDC algorithm performs block-by-block quantization on the large base model. Starting with the first Transformer block of the model, the LQGDC algorithm quantizes and optimizes each block sequentially until the last block. During optimization... When the first block is completed, its input comes from the already optimized first block. The output of the quantization block. This method can take into account the quantization error introduced by the preceding layers and perform real-time correction, thereby improving the final accuracy of the entire model. For the quantization block of the model... Each Transformer block has an optimization process that includes learnable equivalent transformations to adapt to parameter optimization guided by quantization and distribution correction.

[0018] LQGDC first introduces learnable equivalent transformations to adapt to quantization. Traditional quantization methods use static transformations, which cannot adapt to different data distributions. This invention introduces a series of equivalent mathematical transformations with learnable parameters to dynamically reshape the numerical distribution of weights, activation values, and key-value buffers, making them easier to quantize.

[0019] Activation values ​​of large models The existence of "outliers" with huge numerical differences in the data makes it difficult to... Determining a compact quantization range becomes extremely difficult. This invention introduces a learnable weight-activation smoothing module to reduce the difficulty of activation quantization. To enhance the smoothing capability of outliers in weights and activation values, a learnable, channel-specific weight activation smoothing factor is introduced before weight and activation value quantization. Based on the numerical distribution of each activated channel, outliers in the activation values ​​are smoothed into the weights, reducing the difficulty of quantizing the activation values. In the base model, the main calculations come from linear operators. Equivalent transformations are performed on the operators, and learnable parameters are introduced while maintaining the equivalence of the final calculation results. The specific formula is shown below: in, Activate the smoothing factor for the weights. It is a diagonal matrix used to narrow the distribution range of activation values ​​and achieve smooth numerical distribution within the activation channel; These are the input activation values ​​for the model; These are the weights of the model; For bias terms; and and These are the smoothed activation and weight values, which are also the activation and weight values ​​that participate in the subsequent quantization. Input activation value The output obtained after this linear operator. The smoothing factor is then optimized subsequently. , will activate Quantization difficulty transfer to weights The above is because the quantification of weights is usually more robust.

[0020] In the process of weight quantization, in order to constrain the representation range of the weights, the smoothed weights need to be pruned and mapped to the target integer range. The quantization formula is as follows: in These are the quantized weight values; These are the smoothed weight values; This is the scaling factor, used to scale the weights. Zero is used to offset the weights, mapping them to a range of positive integers; The bit width for model quantization; This indicates that the clipping function clips floating-point numbers to integers.

[0021] Even after using the learnable weight-activation smoothing module, a small number of residual extreme values ​​may still exist in the activation and weights. If traditional quantization is used directly... Using values ​​to determine the quantization range wastes a significant number of quantization bits to represent these sparse outliers. This invention introduces a learnable weight-activation pruning module to handle the small number of residual extreme values ​​in the activation and weights.

[0022] This invention introduces a learnable pruning threshold for the weights. A pruning threshold was introduced for the activation value. These thresholds are obtained through Function constraints to This invention allows for dynamic adjustment of the quantization range, thereby retaining most information while ignoring extreme outliers. For weighted quantization pruning, the invention dynamically adjusts the quantization range of the weights using a learnable pruning threshold, reducing quantization pruning error and scaling factor. With zero point The calculation formula is as follows: For activation quantization clipping, this invention uses a learnable clipping threshold. Dynamically adjust the quantization range of the activation value to reduce quantization clipping error and scaling factor. With zero point The calculation formula is as follows: in and This represents the learnable clipping threshold. Control the upper and lower bounds of the weights to define the clipping range. Control the upper and lower bounds of the activation value for pruning, respectively. and Need to pass Function mapping to Within the interval, the formula is as follows: The function can map any real number to... Within the interval, in order to prevent and Exceeding the optimization process The present invention uses this function to measure intervals. and Mapping is performed to ensure the rationality and stability of the pruning threshold.

[0023] The LQGDC algorithm will learn the optimal threshold to achieve the best balance between "truncation error caused by cutting off extreme outliers" and "quantization error caused by retaining a wider range".

[0024] In long text reasoning, the numerical distribution of key-value (KV) caches exhibits drastic variations and outliers across different channels and lexical dimensions, and direct quantization severely compromises the accuracy of attention calculations. To address this issue, this invention introduces inter-channel smoothing and inter-lexical dynamic quantization modules.

[0025] For a linear projection layer, given an input sequence ,in Indicates the length of the sequence, i.e., the number of lexical units; This represents the dimension of the input features. The expression for the linear projection layer is shown in the formula: in This is the weight matrix; Indicates the dimension of the output feature; It is the bias vector; This represents the projected output matrix, i.e., the KV buffer. Here, the weight matrix is... and bias vector The input sequence is subjected to linear transformation and offset adjustment respectively, and the output is generated. It contains feature representations for each word source, which are used for subsequent attention calculations.

[0026] Inter-channel smoothing introduces two learnable parameters: a channel-wise translation parameter. and a scaling parameter The key-value buffer is transformed to scale the numerical range of each channel, narrowing the range of values ​​represented within each channel to facilitate mapping to the corresponding integers after quantization. The channel-level smoothing formula is shown below: in, This indicates that the values ​​in each channel are centered to make the value distribution within each channel more symmetrical. Based on this, scaling parameters are then applied. Channel-level numerical scaling reduces the range of values ​​within a channel, making it easier to map floating-point values ​​to integer values ​​more precisely. This indicates that the translation and scaling parameters can be learned. The output matrix after translation and scaling is then... This can significantly reduce the numerical differences between channels, and further research will focus on... Quantify it.

[0027] Inter-term dynamic quantization on the channel-smoothed matrix The quantization parameters are calculated independently for each token. Specifically, the median is used instead of the mean for centering to enhance robustness to outliers, and a scaling factor is dynamically calculated for each token. The specific formula is shown below: in It is a matrix after channel smoothing, representing the key value information of words; This represents the quantized matrix result, restricting the values ​​within the matrix to a range of bit width. The median of the lexical units is used to center the matrix. It is the KV cache scaling factor, which represents scaling the absolute maximum deviation of the token to the target range; It is the target quantization bit width.

[0028] To balance precision and memory usage, this invention employs a hybrid precision strategy. In autoregressive decoding, only historical key-value pairs are quantized at low bit levels (e.g., 4-bit integers, INT4), while the latest key-value pairs are kept in full precision (16-bit floating-point numbers, FP16). When calculating attention, the dequantized historical cache is concatenated with the current full-precision cache, ensuring accurate capture of the latest information while significantly reducing the storage of historical information.

[0029] After applying the aforementioned learnable transformation to the current block, a precise optimization objective is needed to guide the learnable parameters of all the modules. Updates include: Learnable parameters introduced into the learnable weights-activation smoothing module. Learnable weights - the activation clipping module introduces learnable parameters { The learnable key-value cache channel smoothing and inter-term dynamic quantization modules introduce learnable parameters. This invention introduces distribution-corrected parameter optimization, combining mean square error, cosine similarity, and relative entropy to construct a feature loss function. and attention loss function Composition of composite loss function Used to guide the optimization process: The feature loss function ensures the output of the quantized block It is aligned with the full-precision benchmark in both numerical and semantic directions, and corrects for interlayer cumulative errors.

[0030] The feature loss function is represented at the th... The loss of each Transformer block, where It is the first The output after quantization of a Transformer block It is the first Full-precision output of each Transformer block; From the first The full-precision input of the first Transformer block is used to derive the first... The full-precision output of a Transformer block represents the full-precision output produced by the full-precision block after receiving the output from the previous quantization block.

[0031] First item Full-precision output is measured by mean square error. With quantization output Numerical similarity between them, the second term Ensure full-precision output With quantization output In terms of directional consistency in the feature space, the third and fourth terms perform error correction, actively compensating for and correcting the errors introduced by the previous layer of quantization, breaking the chain of error accumulation layer by layer.

[0032] The attention loss function ensures that the behavior of the quantized attention mechanism is consistent with the original model.

[0033] The attention loss function is represented at the th... The loss of each Transformer block, where This represents the KL divergence, used to measure the difference between two distributions; This represents the full-precision attention output in the i-th Transformer block; This represents the quantized attention output in the i-th Transformer block.

[0034] The attention loss function focuses on restoring the accuracy of attention computation after quantization, and measures the attention output after quantization using KL divergence. and full-precision attention output To correct the deviation and restore the consistency of attention distribution.

[0035] Finally, this invention will perform optimization, with the cloud server using a gradient descent optimizer (such as Adam) to minimize at the block level. and The loss function is used to update the learnable equivalent transformation parameters in the current block via backpropagation. This process is performed sequentially for each Transformer block in the model, when the... After the first block is optimized, its parameters will be fixed. Then, the algorithm will use the output of that quantized block as input to begin optimizing the second block. Each block undergoes the same optimization process, and this process is repeated until all blocks of the model have been optimized, thus completing model quantization.

[0036] In step 4, after all blocks have been optimized, they are assembled into a final quantized subnetwork.

[0037] The validation module on the cloud server performs performance evaluation on the quantized model (e.g., calculating perplexity on a calibration set or evaluating accuracy on standard tasks) to ensure its performance meets standards. Once validation is successful, this highly optimized, lightweight model is distributed back to the requesting edge device. The edge device receives the model and deploys it on its local inference engine (such as ONNX Runtime, TFLite, etc.) to efficiently execute previously complex tasks, enabling low-latency, offline intelligent applications.

[0038] In summary, this invention, through an innovative cloud-edge collaborative framework and a unique distributed correction-guided learnable quantization algorithm, successfully provides personalized, high-performance large models for resource-constrained edge devices, effectively promoting the popularization and application of large-scale AI models at the edge.

[0039] A cloud-edge collaborative large-scale model learnable quantization system for deep-water monitoring includes: at least one edge device for performing task classification, data uploading, and local inference; and a cloud server communicating with the edge device for performing model selection and learnable quantization optimization. a. Edge devices: Equipped with a task evaluation module, a data anonymization and sampling module, and a local inference engine; used for task classification, uploading anonymized calibration data, and executing local inference tasks, respectively; b. Cloud server: configured with a model pool, quantization configuration module, LQGDC quantization execution module and verification module; used to receive monitoring and calibration data, execute the above-mentioned model to learn quantization methods, distribute customized models, and perform performance verification and distribution of optimized quantization models; The task evaluation module determines complexity based on indicators such as task type, input data length (long time series), or required computation. The data desensitization and sampling module is used to sample and desensitize local data to generate a calibration dataset; The model pool stores the base large model; the quantization configuration module performs quantization configuration on the selected model; the LQGDC quantization execution module uses the LQGDC algorithm to quantize the base large model block by block; the verification module performs performance evaluation on the quantized model to ensure that its performance meets the standards. After the verification is passed, the model is distributed back to the edge device that initiated the request.

[0040] The implementation process and methods of the system are the same and will not be described again.

[0041] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) Personalization and high precision: By using real desensitized data of reservoir monitoring uploaded by edge devices, the quantization model generated by this method can dynamically adapt to the data distribution of specific devices and tasks, which is significantly better than the static PTQ method using a general dataset. It can still maintain high precision under low bit quantization, and is especially suitable for the accurate analysis of high-sensitivity water quality indicators.

[0042] (2) Adaptability and flexibility: The learnable parameters (smoothing factor, pruning threshold, etc.) introduced by the distribution correction guide make the quantization process no longer a static mapping, but an optimizable process that can automatically find the best quantization scheme for a specific data distribution.

[0043] (3) Efficient memory compression: A special learnable transformation and mixed precision quantization strategy is designed for KV cache. While ensuring the performance of long context tasks, the memory usage during inference is greatly reduced, effectively supporting the analysis of long time series data.

[0044] (4) Balancing privacy and performance: By using a small amount of anonymized calibration data instead of all local data, user privacy is effectively protected. At the same time, the powerful computing power of the cloud is used to complete a complex quantization optimization process, solving the problem of the cloud-edge information gap. Attached Figure Description

[0045] Figure 1 This is a flowchart illustrating the quantitative aspects of an example of the present invention. Figure 2 This is a diagram of the historical key-value cache structure of an example of the present invention. Detailed Implementation

[0046] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading the present invention, any modifications of the present invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0047] A learnable quantization method and system for a cloud-edge collaborative large-scale model for deep-water monitoring overcomes the accuracy loss problem of traditional quantization methods on edge devices. It achieves dynamic adaptability of quantization parameters to edge data distribution, optimizes KV caching quantization to reduce memory consumption for long text processing, realizes cloud-edge collaboration while protecting privacy, and ensures smooth task execution in scenarios with unstable communication networks. A specific implementation method is provided below: A learnable quantization method for a cloud-edge collaborative large-scale model for deep-water monitoring includes the following steps: Step 1: The edge device assesses the complexity of the local task. When the task complexity is not higher than a preset threshold, the locally deployed lightweight model is invoked for processing, which is sufficient to meet simple daily needs; when the task complexity is higher than the preset threshold, the cloud-based collaborative quantification process is triggered, and Step 2 is executed.

[0048] Edge devices refer to computationally limited equipment such as deep-water buoy control systems, UAV-borne hyperspectral processing units, or shore-based automatic monitoring stations. Complex tasks include, for example, early warning of sudden algal blooms based on air-water collaborative monitoring data, real-time source tracing analysis of pollutant migration paths in reservoir areas, or long-term ecological and environmental trend prediction based on multi-factor coupling. The task evaluation module can determine complexity based on indicators such as task type, input data length (long time series), or required computation. In this case, the lightweight model of the edge device is insufficient, triggering the cloud-based collaborative quantification process and executing step 2.

[0049] Step 2: The data anonymization and sampling module on the edge device randomly samples and anonymizes local monitoring data related to the current complex monitoring task (such as real-time water quality profile data from buoy stations and hyperspectral imagery from drones) to form a calibration dataset. This dataset, along with device hardware configuration information (such as memory size and supported computational precision) and task requirements (such as "algal bloom early warning" and "pollution source tracing"), is sent to the cloud server. Based on the task requirements, the cloud server selects a suitable large-scale model (such as LLaMA-13B or Qwen) from the cloud's model pool, determines the model quantization configuration, and executes the distribution-correction-guided learnable quantization algorithm in step 3.

[0050] Step 3: The cloud server executes a distribution-corrected learnable quantization algorithm. This algorithm introduces learnable equivalent transformations: learnable parameters (such as smoothing factors, pruning thresholds, scaling and translation factors) are introduced into the weights, activation values, and KV cache of the base model to dynamically adjust its numerical distribution, making it more suitable for low-bit quantization. Simultaneously, distribution-corrected parameter optimization is used to construct a composite loss function that combines mean squared error, cosine similarity, and KL divergence. This function optimizes the aforementioned learnable parameters block by block, ensuring that the quantized model output approximates the full-precision model to the greatest extent possible in terms of numerical, directional, and attention distributions. After completing the distribution-corrected parameter optimization, Step 4 is executed.

[0051] The cloud server will utilize calibration datasets uploaded from edge devices to perform an innovative distributed correction-guided learnable quantization algorithm on a selected pedestal large model, generating a low-bit quantization model that highly matches the behavior of the original full-precision model. For example... Figure 1 As shown, the execution flow of this step is as follows: This step begins with initialization, establishing optimization criteria. Before starting quantization optimization, the cloud server first needs to establish an accurate reference baseline for the subsequent optimization process. The cloud server will load the pedestal large model selected in step 2 and the received calibration dataset. Then, the entire calibration dataset is input into the unmodified, full-precision pedestal large model, performing a complete forward propagation. During this process, the system will cache and record the following key information: Block-by-block full-precision output ( For each Transformer block in the model, record its full-precision output tensor after receiving full-precision input. This represents the output of that block under ideal conditions.

[0052] Block-by-block full-precision attention output ( ): Records the attention score matrix for each attention head in the model. This reflects the distribution of association weights between different tokens when the model processes calibration data.

[0053] After initialization, the LQGDC algorithm performs block-by-block quantization on the large base model. Starting with the first Transformer block of the model, the LQGDC algorithm quantizes and optimizes each block sequentially until the last block. During optimization... When the first block is completed, its input comes from the already optimized first block. The output of the quantization block. This method can take into account the quantization error introduced by the preceding layers and perform real-time correction, thereby improving the final accuracy of the entire model. For the quantization block of the model... Each Transformer block has an optimization process that includes learnable equivalent transformations to adapt to parameter optimization guided by quantization and distribution correction.

[0054] LQGDC first introduces learnable equivalent transformations to adapt to quantization. Traditional quantization methods use static transformations, which cannot adapt to different data distributions. This invention introduces a series of equivalent mathematical transformations with learnable parameters to dynamically reshape the numerical distribution of weights, activation values, and key-value buffers, making them easier to quantize.

[0055] Activation values ​​of large models The existence of "outliers" with huge numerical differences in the data makes it difficult to... Determining a compact quantization range becomes extremely difficult. This invention introduces a learnable weight-activation smoothing module to reduce the difficulty of activation quantization. To enhance the smoothing capability of outliers in weights and activation values, a learnable, channel-specific weight activation smoothing factor is introduced before weight and activation value quantization. Based on the numerical distribution of each activated channel, outliers in the activation values ​​are smoothed into the weights, reducing the difficulty of quantizing the activation values. In the base model, the main calculations come from linear operators. Equivalent transformations are performed on the operators, and learnable parameters are introduced while maintaining the equivalence of the final calculation results. The specific formula is shown below: in, Activate the smoothing factor for the weights. It is a diagonal matrix used to narrow the distribution range of activation values ​​and achieve smooth numerical distribution within the activation channel; These are the input activation values ​​for the model; These are the weights of the model; For bias terms; and and These are the smoothed activation and weight values, which are also the activation and weight values ​​used in subsequent quantization. Y is the output obtained by processing the input activation value X through this linear operator. The smoothing factor is then optimized. , will activate Quantization difficulty transfer to weights The above is because the quantification of weights is usually more robust.

[0056] In the process of weight quantization, in order to constrain the representation range of the weights, the smoothed weights need to be pruned and mapped to the target integer range. The quantization formula is as follows: in These are the quantized weight values; These are the smoothed weight values; This is the scaling factor, used to scale the weights. Zero is used to offset the weights, mapping them to a range of positive integers; The bit width for model quantization; This indicates that the clipping function clips floating-point numbers to integers.

[0057] Even after using the learnable weight-activation smoothing module, a small number of residual extreme values ​​may still exist in the activation and weights. If traditional quantization is used directly... Using values ​​to determine the quantization range wastes a significant number of quantization bits to represent these sparse outliers. This invention introduces a learnable weight-activation pruning module to handle the small number of residual extreme values ​​in the activation and weights.

[0058] This invention introduces a learnable pruning threshold for the weights. A pruning threshold was introduced for the activation value. These thresholds are constrained by the Sigmoid function. This invention allows for dynamic adjustment of the quantization range, thereby retaining most information while ignoring extreme outliers. For weighted quantization pruning, the invention dynamically adjusts the quantization range of the weights using a learnable pruning threshold, reducing quantization pruning error and scaling factor. With zero point The calculation formula is as follows: For activation quantization clipping, this invention uses a learnable clipping threshold. Dynamically adjust the quantization range of the activation value to reduce quantization clipping error and scaling factor. With zero point The calculation formula is as follows: in and This represents the learnable clipping threshold. Control the upper and lower bounds of the weights to define the clipping range. Control the upper and lower bounds of the activation value for pruning, respectively. and Need to pass Function mapping to Within the interval, the formula is as follows: The Sigmoid function can map any real number to... Within the interval, in order to prevent { and{ Exceeding the optimization process The present invention uses this function to measure intervals. and{ Mapping is performed to ensure the rationality and stability of the pruning threshold.

[0059] The LQGDC algorithm will learn the optimal threshold to achieve the best balance between "truncation error caused by cutting off extreme outliers" and "quantization error caused by retaining a wider range".

[0060] In long text reasoning, the numerical distribution of key-value (KV) caches exhibits drastic variations and outliers across different channels and lexical dimensions, and direct quantization severely compromises the accuracy of attention calculations. To address this issue, this invention introduces inter-channel smoothing and inter-lexical dynamic quantization modules.

[0061] For a linear projection layer, given an input sequence ,in Indicates the length of the sequence, i.e., the number of lexical units; This represents the dimension of the input features. The expression for the linear projection layer is shown in the formula: in This is the weight matrix; Indicates the dimension of the output feature; It is the bias vector; This represents the projected output matrix, i.e., the KV buffer. Here, the weight matrix is... and bias vector The input sequence is subjected to linear transformation and offset adjustment respectively, and the output is generated. It contains feature representations for each word source, which are used for subsequent attention calculations.

[0062] Inter-channel smoothing introduces two learnable parameters: a channel-wise translation parameter. and a scaling parameter The key-value buffer is transformed to scale the numerical range of each channel, narrowing the range of values ​​represented within each channel to facilitate mapping to the corresponding integers after quantization. The channel-level smoothing formula is shown below: in, This indicates that the values ​​in each channel are centered to make the value distribution within each channel more symmetrical. Based on this, scaling parameters are then applied. Channel-level numerical scaling reduces the range of values ​​within a channel, making it easier to map floating-point values ​​to integer values ​​more precisely. This indicates that the translation and scaling parameters can be learned. The output matrix after translation and scaling is then... This can significantly reduce the numerical differences between channels, and further research will focus on... Quantify it.

[0063] Inter-term dynamic quantization on the channel-smoothed matrix The quantization parameters are calculated independently for each token. Specifically, the median is used instead of the mean for centering to enhance robustness to outliers, and a scaling factor is dynamically calculated for each token. The specific formula is shown below: in It is a matrix after channel smoothing, representing the key value information of words; This represents the quantized matrix result, restricting the values ​​within the matrix to a range of bit width. The median of the lexical units is used to center the matrix. It is the KV cache scaling factor, which represents scaling the absolute maximum deviation of the token to the target range; It is the target quantization bit width.

[0064] To balance precision and memory usage, this invention employs a hybrid precision strategy, such as... Figure 2 As shown, in autoregressive decoding, only historical key-value pairs are quantized at low bit depth (e.g., 4-bit integers, INT4), while the latest key-value pairs are kept in full precision (16-bit floating-point numbers, FP16). When calculating attention, the dequantized historical cache is concatenated with the current full-precision cache, which ensures accurate capture of the latest information while significantly compressing the storage of historical information.

[0065] After applying the aforementioned learnable transformation to the current block, a precise optimization objective is needed to guide the learnable parameters of all the modules. Updates include: Learnable parameters introduced into the learnable weights-activation smoothing module. Learnable weights - the activation clipping module introduces learnable parameters { The learnable key-value cache channel smoothing and inter-term dynamic quantization modules introduce learnable parameters. This invention introduces distribution-corrected parameter optimization, combining mean square error, cosine similarity, and relative entropy to construct a feature loss function. and attention loss function Composition of composite loss function Used to guide the optimization process: The feature loss function ensures the output of the quantized block It is aligned with the full-precision benchmark in both numerical and semantic directions, and corrects for interlayer cumulative errors.

[0066] The feature loss function is represented at the th... The loss of each Transformer block, where It is the first The output after quantization of a Transformer block It is the first Full-precision output of each Transformer block; From the first The full-precision input of the first Transformer block is used to derive the first... The full-precision output of a Transformer block represents the full-precision output produced by the full-precision block after receiving the output from the previous quantization block.

[0067] First item Full-precision output is measured by mean square error. With quantization output Numerical similarity between them, the second term Ensure full-precision output With quantization output In terms of directional consistency in the feature space, the third and fourth terms perform error correction, actively compensating for and correcting the errors introduced by the previous layer of quantization, breaking the chain of error accumulation layer by layer.

[0068] The attention loss function ensures that the behavior of the quantized attention mechanism is consistent with the original model.

[0069] The attention loss function is represented at the th... The loss of each Transformer block, where This represents the KL divergence, used to measure the difference between two distributions; Indicates the first Full-precision attention output in each Transformer block; Indicates the first The quantized attention output in each Transformer block.

[0070] The attention loss function focuses on restoring the accuracy of attention computation after quantization, and measures the attention output after quantization using KL divergence. and full-precision attention output To correct the deviation and restore the consistency of attention distribution.

[0071] Finally, this invention will perform optimization, with the cloud server using a gradient descent optimizer (such as Adam) to minimize at the block level. and The loss function is used to update the learnable equivalent transformation parameters in the current block via backpropagation. This process is performed sequentially for each Transformer block in the model, when the... After the first block is optimized, its parameters will be fixed. Then, the algorithm will use the output of that quantized block as input to begin optimizing the second block. Each block undergoes the same optimization process, and this process is repeated until all blocks of the model have been optimized, thus completing model quantization.

[0072] Step 4: After all blocks are optimized, they are assembled into a final quantized subnetwork. The validation module on the cloud server performs performance evaluation on the quantized model (e.g., calculating perplexity on a calibration set or evaluating accuracy on standard tasks) to ensure its performance meets the standards. After successful validation, this highly optimized, lightweight model is distributed back to the requesting edge device. Upon receiving the model, the edge device deploys it on its local inference engine (such as ONNXRuntime, TFLite, etc.) to efficiently perform the previously complex tasks, enabling low-latency, offline intelligent analysis and security early warning applications of air-water collaborative monitoring data.

[0073] System Implementation Example: This invention also provides a learnable quantization system for a cloud-edge collaborative large model for deep-water monitoring, comprising: Edge monitoring devices (EDs), such as deep-water buoy control systems and unmanned aerial vehicle (UAV) systems, are equipped with task evaluation modules, data anonymization and sampling modules, and local inference engines; they are used to perform water environment task classification, upload anonymized calibration data, and perform local inference tasks.

[0074] The cloud server is equipped with a model pool, a quantization configuration module, an LQGDC quantization execution module, and a verification module; it is used to receive monitoring and calibration data, perform learnable quantization optimization, and distribute customized models.

[0075] Obviously, those skilled in the art should understand that the modules of the above-described embodiments of the present invention, "A Learnable Quantization Method and System for a Cloud-Edge Collaborative Large Model for Deepwater Monitoring," can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using device-executable program code, and thus, can be stored in a storage device for execution by the computing device. Therefore, the embodiments of the present invention are not limited to any specific hardware and software combination.

Claims

1. A learnable quantization method for a cloud-edge collaborative large-scale model for deep-water monitoring, characterized in that, Includes the following steps: Step 1) Task complexity assessment and triggering: When the task complexity is not higher than the preset threshold, the locally deployed model is invoked for processing; when the task complexity is higher than the preset threshold, step 2) is executed. Step 2) Data Upload and Model Determination: The edge device samples and de-identifies local data to generate a calibration dataset, and uploads the calibration dataset, along with the device hardware information and task requirements, to the cloud server; The cloud server selects a large base model from the model pool in the cloud based on task requirements and determines the model quantization configuration; Step 3) The cloud server executes a distribution-corrected learnable quantization algorithm, which introduces a learnable equivalent transformation: learnable parameters are introduced into the weights, activation values, and KV cache of the base model to dynamically adjust the numerical distribution of weights, activation values, and KV cache to adapt to the dynamically changing distribution of reservoir monitoring data; at the same time, distribution-corrected parameter optimization is used to construct a composite loss function, which combines mean square error, cosine similarity, and KL divergence to optimize the learnable parameters block by block to obtain the quantization model; Step 4) The cloud server performs performance verification on the quantization model and distributes the optimized quantization model to the corresponding edge devices. The edge devices load the model and execute water environment monitoring inference tasks.

2. The learnable quantization method for a cloud-edge collaborative large model for deep-water monitoring according to claim 1, characterized in that, In step 1), edge devices refer to deep-water buoy control systems, UAV-borne hyperspectral processing units, or shore-based automatic monitoring stations; tasks with a complexity higher than a preset threshold refer to monitoring tasks that require the base model to perform, including: early warning of sudden algal blooms and source tracing analysis of pollutants; the complexity is determined by the task evaluation module based on the task type, input data length, or required computational load.

3. The learnable quantization method for a cloud-edge collaborative large model for deep-water monitoring according to claim 1, characterized in that, In step 3), the cloud server uses the calibration dataset uploaded by the edge device to perform a distribution correction-guided learnable quantization algorithm on the selected large base model. The algorithm is first initialized and optimization criteria are established. The cloud server will load the pedestal large model selected in step 2) and the received calibration dataset, and then input the entire calibration dataset into the unmodified full-precision pedestal large model to perform a complete forward propagation.

4. The learnable quantization method for a cloud-edge collaborative large model for deep-water monitoring according to claim 3, characterized in that, During the forward propagation process, the cloud server caches and records the following information: Block-by-block full-precision output For each Transformer block in the model, record the full-precision output tensor it produces after receiving full-precision input; Block-by-block full-precision attention output : Records the attention score matrix for each attention head in the model.

5. The learnable quantization method for a cloud-edge collaborative large model for deep-water monitoring according to claim 3, characterized in that, After initialization, the LQGDC algorithm performs block-by-block quantization on the large base model. Starting with the first Transformer block of the model, the LQGDC algorithm quantizes and optimizes each block sequentially until the last block. During optimization... When the first block is completed, its input comes from the already optimized first block. The output of the quantization block; for the quantization block in the model Each Transformer block has an optimization process that includes learnable equivalent transformations to adapt to parameter optimization guided by quantization and distribution correction.

6. The learnable quantization method for a cloud-edge collaborative large model for deep-water monitoring according to claim 5, characterized in that, The LQGDC algorithm introduces learnable equivalent transformations to adapt to quantization, dynamically reshaping the numerical distribution of weights, activation values, and key-value buffers; specifically including: To enhance the smoothing capability of outliers in weights and activation values, a learnable, channel-based weight activation smoothing factor is introduced before weight and activation value quantization. Based on the distribution of activation channel values, outliers in the activation values ​​are smoothed into the weights, reducing the difficulty of quantizing activation values. In the base model, equivalent transformations are performed on linear operators, and learnable parameters are introduced while maintaining the equivalence of the final calculation results. The specific formula is shown below: in, Activate the smoothing factor for the weights. It is a diagonal matrix used to narrow the distribution range of activation values ​​and achieve smooth numerical distribution within the activation channel; These are the input activation values ​​for the model; These are the weights of the model; For bias terms; and and These are the smoothed activation and weight values, which are also the activation and weight values ​​used in subsequent quantization; the smoothing factor is then optimized. , will activate Quantization difficulty transfer to weights superior; In the weight quantization process, in order to constrain the representation range of the weights, the smoothed weights need to be pruned and mapped to the target integer range, introducing a learnable pruning threshold for the weights and activation values. and These thresholds are obtained through Function constraints to The quantization range of the weights is dynamically adjusted through a learnable pruning threshold to reduce quantization pruning errors; for weight quantization pruning, a learnable pruning threshold is used. Dynamically adjust the quantization range of weight values ​​to reduce quantization clipping error and scaling factor. With zero point The calculation formula is as follows: For activation quantization clipping, a learnable clipping threshold is used. Dynamically adjust the quantization range of the activation value to reduce quantization clipping error and scaling factor. With zero point The calculation formula is as follows: in and This represents the learnable clipping threshold. Control the upper and lower bounds of the weights to define the clipping range. Control the upper and lower bounds of the activation value for pruning, respectively. and Need to pass Function mapping to Within the interval, the formula is as follows: The function can map any real number to... Within the interval, in order to prevent and Exceeding the limits during subsequent optimization processes The present invention uses this function to measure intervals. and Perform mapping; Inter-channel smoothing introduces two learnable parameters: a channel-wise translation parameter. and a scaling parameter The key-value buffer is transformed to scale the value range of each channel, reducing the range of values ​​represented within each channel to facilitate mapping to the corresponding integers after quantization. The channel-level smoothing formula is shown below: in, This indicates that the values ​​for each channel are centered, and then scaled using parameters. Perform channel-level numerical scaling; This represents the learnable translation and scaling parameters, and the output matrix after translation and scaling processing. To reduce the numerical differences between channels, the following will be addressed. Quantify; Inter-term dynamic quantization on the channel-smoothed matrix The quantization parameters are calculated independently for each token; specifically, the median is used for centering, and the scaling factor is dynamically calculated for each token. The specific formula is as follows: in It is a matrix after channel smoothing, representing the key value information of words; This represents the quantized matrix result, restricting the values ​​within the matrix to a range of bit width. The median of the lexical units is used to center the matrix. It is the KV cache scaling factor, which represents scaling the absolute maximum deviation of the token to the target range; It is the target quantization bit width; In autoregressive decoding, only historical key-value pairs are quantized at low bit depth, while the latest key-value pair is kept at full precision. When calculating attention, the dequantized historical cache is concatenated with the current full-precision cache.

7. The learnable quantization method for a cloud-edge collaborative large model for deep-water monitoring according to claim 1, characterized in that, The distribution-corrected parameter optimization combines mean squared error, cosine similarity, and relative entropy to construct a feature loss function. and attention loss function Composition of composite loss function Used to guide the optimization process: The feature loss function ensures the output of the quantized block Aligned with a full-precision benchmark in both numerical and semantic dimensions, and corrected for interlayer cumulative errors; The feature loss function is represented at the th... The loss of each Transformer block, where It is the first The output after quantization of a Transformer block It is the first Full-precision output of each Transformer block; From the first The full-precision input of the first Transformer block is used to derive the first... The full-precision output of a Transformer block represents the full-precision output produced by the full-precision block after receiving the output from the previous quantization block; First item Full-precision output is measured by mean square error. With quantization output Numerical similarity between them, the second term Ensure full-precision output With quantization output For orientation consistency in feature space, the third and fourth terms have been corrected for errors; The attention loss function ensures that the behavior of the quantized attention mechanism is consistent with the original model; The attention loss function is represented in the th case. The loss of each Transformer block, where This represents the KL divergence, used to measure the difference between two distributions; Indicates the first Full-precision attention output in each Transformer block; This represents the quantized attention output in the i-th Transformer block; The attention loss function focuses on restoring the accuracy of attention computation after quantization, and measures the attention output after quantization using KL divergence. and full-precision attention output To correct the deviation and restore the consistency of attention distribution.

8. The learnable quantization method for a cloud-edge collaborative large model for deep-water monitoring according to claim 7, characterized in that, Cloud servers use gradient descent optimizers to minimize blocks. and The loss function is used to update the learnable equivalent transformation parameters in the current block via backpropagation. This process is performed sequentially for each Transformer block in the model, when the... After the first block is optimized, its parameters will be fixed; then, the output of that quantization block will be used as input to begin optimizing the second block. Each block undergoes the same optimization process, and this process is repeated until all blocks of the model have been optimized, thus completing model quantization.

9. The learnable quantization method for a cloud-edge collaborative large model for deep-water monitoring according to claim 1, characterized in that, In step 4, after all blocks are optimized, they are assembled into a final quantization sub-network. The verification module on the cloud server performs a performance evaluation on the quantization model to ensure that its performance meets the standards. After the verification is passed, the model is distributed back to the edge device that initiated the request. After receiving the model, the edge device deploys it on its local inference engine to perform the complex tasks mentioned earlier.

10. A cloud-edge collaborative large-scale model learnable quantization system for deep-water monitoring, characterized in that, include: At least one edge device is used to perform task grading, data uploading, and local inference; A cloud server, communicating with the edge device, is used to perform model selection and learnable quantization optimization; Edge devices are equipped with a task evaluation module, a data anonymization and sampling module, and a local inference engine; these are used to perform task classification, upload anonymized calibration data, and execute local inference tasks, respectively. Cloud server: configured with model pool, quantization configuration module, LQGDC quantization execution module and verification module; It is used to receive monitoring and calibration data, execute the above model to learn quantization methods, distribute customized models, and perform performance verification and distribution on the optimized quantization models; The task evaluation module determines complexity based on indicators such as task type, input data length, or required computation. The data desensitization and sampling module is used to sample and desensitize local data to generate a calibration dataset; The model pool stores the base large model; The quantization configuration module configures the selected model for quantization; the LQGDC quantization execution module uses the LQGDC algorithm to quantize the large base model block by block; the verification module evaluates the performance of the quantized model to ensure that its performance meets the standards. After verification, the model is distributed back to the edge device that initiated the request.

Images