Source coding matrix data processing method based on modified conjugate gradient algorithm

Abstract

This invention relates to the field of computer technology and discloses a method for processing source coding matrix data based on a modified conjugate gradient algorithm. This method integrates sparsity constraints and fixed-point quantization mechanisms in iterative optimization. Through gradient masking, modified conjugate parameter calculation, and Armijo criterion line search, and by synchronously performing hard-threshold sparsification and fixed-point quantization after each update, it directly generates a sparse fixed-point coding matrix that meets hardware deployment requirements. This invention achieves end-to-end alignment between coding matrix optimization and hardware deployment requirements by embedding sparsity constraints and fixed-point quantization mechanisms within the iterative process of the modified conjugate gradient algorithm.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, specifically relating to a source coding matrix data processing method based on the modified conjugate gradient algorithm. Background Technology

[0002] With the widespread application of large-scale source coding in communication, compressed sensing, and neural network model compression, the design and optimization of coding matrices have become crucial for improving system performance. These coding matrices typically exhibit high-dimensional density, and the optimization process involves massive floating-point operations and storage overhead, posing significant challenges to computational resources during iterative solutions. Simultaneously, deployment requirements for embedded devices or real-time inference scenarios necessitate a highly sparse structure and low-precision representation capabilities in the final coding matrix to reduce power consumption and latency in hardware implementations. Current mainstream optimization methods are mostly performed in the full-precision floating-point domain, lacking explicit modeling of hardware deployment constraints, leading to a disconnect between optimization results and actual deployment goals.

[0003] While matrix optimization methods based on gradient descent or conjugate gradient algorithms offer advantages in convergence and numerical stability, their standard forms do not consider sparsity-induced mechanisms or precision-aware computational paths. Traditional strategies often employ post-processing to compress the optimized dense matrix. This two-stage process not only compromises the integrity of the original optimization objective but can also significantly degrade encoding performance. Furthermore, existing methods use a uniform precision and fully connected computation mode for core operations such as matrix-vector multiplication during iteration, failing to dynamically adapt to the varying importance of different data blocks, resulting in wasted computational resources and inefficient sparse structure generation.

[0004] Existing technologies for source coding matrix optimization suffer from the following problems: a conflict between high-dimensional dense optimization and hardware-friendly sparse structures; a representation gap between continuous floating-point optimization and low-precision discrete deployment; and an imbalance in resource matching between fixed computational granularity and dynamic data importance. These problems are particularly pronounced in resource-constrained edge computing or real-time communication systems, making it difficult to synergistically improve coding efficiency, hardware energy efficiency, and system response speed. Therefore, there is an urgent need for a source coding matrix design method that can deeply integrate sparsity-induced optimization, mixed-precision computation, and iterative optimization processes to achieve a balance between high performance and high hardware efficiency. Summary of the Invention

[0005] This invention provides a source coding matrix data processing method based on a modified conjugate gradient algorithm, aiming to solve the technical problems of huge computational and storage overhead in the existing coding matrix optimization process, and the disconnect between the optimization results and the deployment requirements of embedded or real-time systems. The method constructs a modified conjugate gradient optimization framework that integrates sparsity constraints and low-precision quantization mechanisms. During the iteration process, it simultaneously achieves the sparsification of the matrix structure and the fixed-point representation of the numerical representation, thereby directly generating a coding matrix that meets hardware deployment requirements. This avoids the performance loss and process fragmentation caused by the additional pruning and quantization after traditional floating-point optimization.

[0006] This invention provides a source coding matrix data processing method based on the modified conjugate gradient algorithm, including: initializing the dimension as... OK The column encoding matrix, The number of source symbols, The encoding vector dimension is defined; the initial search direction is set to the negative gradient direction, and the upper limit of the number of iterations, the convergence threshold, the target sparsity ratio, and the fixed-point quantization bit width are also set.

[0007] In each iteration, the gradient matrix of the current encoding matrix under the target loss function is calculated; a gradient masking operation based on the current sparse mask is applied to the gradient matrix to freeze the gradient components corresponding to the zeroed elements;

[0008] The conjugate parameters are calculated based on the modified conjugate gradient formula. These conjugate parameters are adjusted by introducing a gradient orthogonality correction term and a sparsity adaptive decay factor to suppress the search for ineffective directions and accelerate convergence.

[0009] Update the search direction according to the conjugate parameter; perform a one-dimensional line search along the search direction to determine the optimal step size, which must satisfy the Armijo-Goldstein criterion and trigger sparse projection and fixed-point quantization operations immediately after the update;

[0010] The encoding matrix is ​​updated using the optimal step size, and a hard thresholding sparsification operation is performed on the updated encoding matrix, retaining the top values ​​with the largest absolute values. One element is set to zero, and the rest are forced to zero.

[0011] Fixed-point quantization is performed on the sparsified encoding matrix, mapping each non-zero element to a discrete set of values ​​defined by the fixed-point quantization bit width. This discrete set of values ​​consists of sign bits, integer bits, and fractional bits, and its dynamic range and resolution are determined by the statistical extreme values ​​in the early stage of training.

[0012] Update the sparse mask to record the current non-zero element position; determine whether the current iteration meets the convergence condition or reaches the upper limit of the number of iterations. If it meets the condition, output the final sparse fixed-point encoding matrix; otherwise, proceed to the next iteration.

[0013] Preferably, the target loss function is the mean square error function of the source reconstruction error, and its input is the original source data and the data reconstructed by the current coding matrix and the fixed decoder. The fixed decoder is a preset pseudo-inverse matrix or a learned decoding mapping function.

[0014] Preferably, applying a gradient masking operation based on the current sparse mask to the gradient matrix includes: setting the gradient matrix elements corresponding to positions with a value of 0 in the current sparse mask to zero, and retaining only the gradient components corresponding to positions with a value of 1 in the mask to participate in subsequent conjugate parameter calculation and search direction update.

[0015] Preferably, the conjugate parameter The calculation uses the following modified formula: , For the first The gradient vector obtained by vectorizing the gradient matrix of the next iteration. For the first The gradient vector obtained by vectorizing the gradient matrix of the next iteration. Indicates transpose. For the first The search direction vector for the next iteration. This is a sparsity adaptive decay factor, dynamically adjusted according to the difference between the current sparsity and the target sparsity. When the current sparsity is less than the target sparsity, Positive values ​​are used to enhance the gradient orthogonality constraint; when the current sparsity reaches or exceeds the target sparsity, Set to zero.

[0016] Preferably, a one-dimensional line search is performed along the search direction to determine the optimal step size, including: the optimal step size of the one-dimensional line search. The step size is determined using a backtracking method, with an initial step size of 1 and a reduction factor of 0.5 each time, until the condition is met. , For loss function, For the first The encoding matrix of the next iteration For the first The search direction for the next iteration. is the Armijo condition constant.

[0017] Preferably, during the hard thresholding sparsity operation on the updated encoding matrix, The calculation method is as follows , The target sparsity ratio, This is the floor function.

[0018] Preferably, in the fixed-point quantization operation performed on the sparsified coding matrix, the fixed-point quantization bit width is 8 bits, including 1 sign bit, 3 integer bits, and 4 fractional bits, and the quantization step size is... , and These are the historical maximum and minimum values ​​of all non-zero elements in the current encoding matrix during the first 100 iterations of training.

[0019] Preferably, after each iteration of sparsification and quantization operations, a numerical stability check is performed on the encoding matrix. If any non-zero elements exceed the fixed-point representation range, a range recalibration mechanism is triggered, and the matrix is ​​recalculated. and And update the quantization parameters.

[0020] Preferably, the sparse mask is updated in each iteration to an indicator matrix of the non-zero element positions in the current encoding matrix, with the same dimension as the encoding matrix, the non-zero element positions corresponding to a value of 1, and the rest being 0.

[0021] Preferably, the convergence condition is that the decrease in the loss function over three consecutive iterations is less than a preset convergence threshold.

[0022] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0023] 1. This invention achieves end-to-end alignment between encoding matrix optimization and hardware deployment requirements by embedding sparsity constraints and fixed-point quantization mechanisms in the iterative process of the modified conjugate gradient algorithm.

[0024] 2. This invention performs sparse projection and fixed-point quantization immediately after each iteration update, so that the optimization process is always carried out in the target representation space, avoiding the performance collapse problem that occurs after floating-point optimization results are quantized.

[0025] 3. The modified conjugate gradient formula proposed in this invention introduces a sparsity adaptive decay factor and a gradient orthogonality correction term, which can maintain the effectiveness of the search direction under high sparsity conditions and overcome the problem that the traditional conjugate gradient method is prone to divergence or slow convergence under non-smooth and non-convex sparse constraints.

[0026] 4. By combining hard thresholding with fixed-point quantization, an encoding matrix that meets the storage and computing resource limitations of embedded systems is directly generated, reducing the engineering complexity and performance tuning costs during the deployment phase. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the overall technical solution architecture of the present invention;

[0028] Figure 2This is a schematic diagram of the core principle framework of the modified conjugate gradient algorithm in this invention, which integrates sparsity constraints and fixed-point quantization mechanism;

[0029] Figure 3 This is a flowchart illustrating the logical process of gradient masking, search direction updating, and step size determination during the iterative optimization of the encoding matrix in this invention.

[0030] Figure 4 This is a logical flow diagram of the collaborative execution stage of sparse projection and fixed-point quantization in this invention;

[0031] Figure 5 This is a schematic diagram of the multi-level interaction relationship and data flow between the coding matrix optimization system and external information sources and decoders in this invention. Detailed Implementation

[0032] This invention provides a source coding matrix data processing method based on a modified conjugate gradient algorithm. It directly embeds sparsity constraints and fixed-point quantization mechanisms into the iterative process of conjugate gradient optimization, enabling the coding matrix to simultaneously satisfy the dual constraints of structural sparsity and low-precision numerical representation imposed by hardware deployment during optimization. This method is applicable to resource-constrained scenarios such as source compression, embedded signal processing, and edge computing, where the coding matrix needs to achieve efficient source reconstruction under limited storage and computing power. The following combines... Figures 1 to 5 The following description elaborates on specific embodiments of the present invention.

[0033] The method first initializes the dimension as follows: OK Column encoding matrix , The number of source symbols, This is the dimension of the encoding vector. This encoding matrix is ​​used to encode the original source data. Mapping to a low-dimensional coding space ,Right now Then through a fixed decoder (For example, a pre-defined pseudo-inverse matrix or a learned decoding mapping function) Reconstruct the information source Target loss function Defined as the mean square error of the source reconstruction error, i.e. The objective loss function measures the current encoding matrix. In fixed decoder The reconstruction performance is the basis for subsequent gradient calculation and optimization iteration.

[0034] Initialize the encoding matrix Then, several key hyperparameters are set: upper limit of iteration count. The value is 500; the convergence threshold. The value is 10. -6Target sparsity ratio The value ranges from 0.01 to 0.5, with a typical value of 0.2; fixed-point quantization bit width. The value is 8 bits, including 1 sign bit, 3 integer bits, and 4 decimal bits. Simultaneously, the initial search direction is initialized. The direction of the negative gradient is... , For the loss function in The gradient matrix at that point. Initialize the sparse mask. For a one matrix, the dimension is 1. Consistency is used to record the position of the current non-zero element.

[0035] Entering the main iteration loop, for the... The next iteration ( (Counting from zero), first calculate the current encoding matrix. (No. The encoding matrix of the next iteration in the target loss function The gradient matrix below Each element of the gradient matrix The loss function represents the loss function applied to the encoding matrix. Line number Column elements The partial derivatives are calculated using the chain rule and The Jacobian matrix. Because As a fixed structure, its gradient propagation path remains unchanged in each iteration, thus ensuring the determinism and reproducibility of gradient calculation.

[0036] get Then, a gradient masking operation is performed. This gradient masking operation utilizes the current sparse mask. The gradient elements corresponding to positions with a value of 0 in the mask are set to zero, and only the gradient components corresponding to positions with a value of 1 are retained for subsequent calculations. The masked gradient matrix. , This indicates element-wise multiplication. This operation ensures that elements of the encoded matrix that have been set to zero are not activated in subsequent updates, thereby maintaining the stability of the sparse structure and avoiding invalid gradients from interfering with the generation of search directions.

[0037] Calculate conjugate parameters based on the modified conjugate gradient formula The traditional conjugate gradient method uses the Fletcher-Reeves or Polak-Ribière formulas to calculate... However, under high sparsity and non-smooth constraints, the search direction is prone to failure. This invention introduces a gradient orthogonality correction term and a sparsity adaptive decay factor, constructing the following correction formula:

[0038] ;

[0039] For the first The gradient vector obtained by vectorizing the gradient matrix of the next iteration. For the first The gradient vector obtained by vectorizing the gradient matrix of the next iteration. Indicates transpose. For the first The search direction vector for the next iteration. This is a sparsity adaptive decay factor, dynamically adjusted according to the difference between the current sparsity and the target sparsity. When the current sparsity is less than the target sparsity, Positive values ​​are used to enhance the gradient orthogonality constraint; when the current sparsity reaches or exceeds the target sparsity, Set to zero.

[0040] The design enhances gradient orthogonality constraints when sparsity is not met, suppresses the accumulation of ineffective directions, and accelerates convergence to the target sparse structure; after sparsity is met, it degenerates into the standard conjugate gradient form to maintain convergence stability.

[0041] Calculated Update search direction: . For the first The search direction for the next iteration. For the first The search direction for the next iteration. By combining the current negative gradient information with the conjugate information of historical search directions, global convergence efficiency is improved while maintaining descent properties.

[0042] After determining the search direction, perform a one-dimensional line search to determine the optimal step size. This invention employs a backtracking method, with an initial step size of 1 and a reduction factor of 0.5 each time, until the Armijo-Goldstein criterion is satisfied:

[0043] ;

[0044] is the Armijo condition constant.

[0045] The Armijo-Goldstein criterion ensures that each update brings a sufficient decrease in loss, avoiding oscillations or divergence caused by excessively large step sizes. Once a condition is found... That is, use it to update the encoding matrix. .

[0046] Updated encoding matrix Immediately trigger sparse projection and fixed-point quantization operations, instead of continuing iteration in the floating-point domain. First, perform hard-threshold sparsity reduction: calculate... That is, the total number of non-zero elements; for Sort all elements in descending order of absolute value, keeping the first few elements. Find the largest element, and force the rest of the elements to zero to obtain a sparse matrix. This operation ensures that the encoding matrix strictly meets the preset sparsity ratio and retains the elements that contribute the most to the reconstruction performance.

[0047] right Perform fixed-point quantization. Before quantization, the dynamic range needs to be determined. During the initial training phase (the first 100 iterations), continuously record the historical maximum values ​​of all non-zero elements. and minimum value Quantization step size Since the sign bit occupies 1 bit and the significant value bits are 7 bits in an 8-bit fixed-point representation, the number of discrete levels is 2^3. 7 .right Each non-zero element Perform the following mapping:

[0048]

[0049] This is a rounding function. This is the quantization function. The quantized value is restricted to... In the discrete set formed, it is ensured that all non-zero elements can be accurately represented by an 8-bit fixed-point format.

[0050] After sparsification and quantization are completed, the sparse mask is updated. If and only if the first In the encoding matrix of the nth iteration, the th line, number Column elements Otherwise, it is 0. This sparse mask will be used in the gradient masking step of the next iteration, forming a closed-loop feedback.

[0051] After each quantization, perform a numerical stability check: traverse All non-zero elements in the set, if they exist or If this happens, the range recalibration mechanism is triggered. The maximum and minimum values ​​of all non-zero elements are recalculated, and the range is updated. and And recalculate This mechanism prevents quantization overflow caused by insufficient early extreme value estimation, thus ensuring the integrity of the numerical representation.

[0052] Finally, determine the convergence condition: if the loss function decreases by a certain amount in three consecutive iterations... All are less than the convergence threshold If the iteration terminates, or if The iteration terminates when the condition is met. The final sparse fixed-point coding matrix is ​​then output. This sparse fixed-point coding matrix can be directly deployed in embedded systems without additional pruning or quantization post-processing.

[0053] At the system level, this invention corresponds to a source coding matrix data processing system based on a modified conjugate gradient algorithm. This system includes a coding matrix initialization module for generating... dimensional initial matrix; parameter configuration module, used to set The module includes hyperparameters; a gradient calculation module that receives the current encoding matrix and the original source data, and outputs the gradient matrix; a gradient masking module that receives the gradient matrix and a sparse mask, and performs element-wise masking; and a conjugate parameter calculation module that implements the above corrections. The system includes: a formula; a search direction update module that performs linear combination of vectors; an optimal step size determination module that performs backtracking search; an encoding matrix update module that performs matrix subtraction; a sparsity processing module that performs hard threshold selection; a fixed-point quantization module that performs discrete mapping; a sparse mask update module that generates a new mask; a numerical stability verification module that monitors quantization overflow and triggers recalibration; and a convergence judgment module that controls the termination of iteration.

[0054] The modules are connected via an internal data bus, forming a closed-loop optimized pipeline. The raw source data y and the fixed decoder... As an external input, it does not participate in the optimization process and is only used for loss calculation and gradient backpropagation. The system runs on a general-purpose processor or a dedicated accelerator, and all operations are performed in fixed-point or mixed precision to ensure consistency with the deployment environment.

[0055] This embodiment ensures that the optimized trajectory always lies within the target representation space by forcing sparse projection and fixed-point quantization in each iteration, thus avoiding the performance collapse of floating-point optimization results after quantization, a problem common in traditional two-stage methods. The modified sparsity adaptive decay factor in the conjugate gradient formula alleviates the problem of search direction degradation under high sparsity, guaranteeing both convergence speed and stability.

[0056] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0057] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for processing source coding matrix data based on the modified conjugate gradient algorithm, characterized in that, include: initializing a coding matrix with dimensions of rows and columns, wherein n is the number of source symbols, wherein m is the dimension of the coding vector; setting an initial search direction as a negative gradient direction, and setting an upper limit of iteration times, a convergence threshold, a target sparsity ratio, and a fixed-point quantization bit width; In each iteration, the gradient matrix of the current encoding matrix under the target loss function is calculated; Apply a gradient masking operation based on the current sparse mask to the gradient matrix to freeze the gradient components corresponding to the zeroed elements; The conjugate parameters are calculated based on the modified conjugate gradient formula. These conjugate parameters are adjusted by introducing a gradient orthogonality correction term and a sparsity adaptive decay factor to suppress the search for ineffective directions and accelerate convergence. Update the search direction according to the conjugate parameter; perform a one-dimensional line search along the search direction to determine the optimal step size, which must satisfy the Armijo-Goldstein criterion and trigger sparse projection and fixed-point quantization operations immediately after the update; The encoding matrix is ​​updated using the optimal step size, and a hard thresholding sparsification operation is performed on the updated encoding matrix, retaining the top values ​​with the largest absolute values. One element is set to zero, and the rest are forced to zero. Fixed-point quantization is performed on the sparsified encoding matrix, mapping each non-zero element to a discrete set of values ​​defined by the fixed-point quantization bit width. This discrete set of values ​​consists of sign bits, integer bits, and fractional bits, and its dynamic range and resolution are determined by the statistical extreme values ​​in the early stage of training. Update the sparse mask to record the current non-zero element position; determine whether the current iteration meets the convergence condition or reaches the upper limit of the number of iterations. If it meets the condition, output the final sparse fixed-point encoding matrix; otherwise, proceed to the next iteration. Applying a gradient masking operation based on the current sparse mask to the gradient matrix includes: setting the gradient matrix elements corresponding to the positions with a value of 0 in the current sparse mask to zero, and retaining only the gradient components corresponding to the positions with a value of 1 in the mask to participate in subsequent conjugate parameter calculation and search direction update; The conjugate parameter The calculation uses the following modified formula: , For the first The gradient vector obtained by vectorizing the gradient matrix of the next iteration. For the first The gradient vector obtained by vectorizing the gradient matrix of the next iteration. Indicates transpose. For the first The search direction vector for the next iteration. This is a sparsity adaptive decay factor, dynamically adjusted according to the difference between the current sparsity and the target sparsity. When the current sparsity is less than the target sparsity, Positive values ​​are used to enhance the gradient orthogonality constraint; when the current sparsity reaches or exceeds the target sparsity, Set to zero; During the hard thresholding sparsification operation on the updated encoding matrix, The calculation method is as follows , The target sparsity ratio, This is the floor function.

2. The source coding matrix data processing method based on the modified conjugate gradient algorithm according to claim 1, characterized in that, The target loss function is the mean square error function of the source reconstruction error. Its input is the original source data and the data reconstructed by the current coding matrix and the fixed decoder. The fixed decoder is a preset pseudo-inverse matrix or a learned decoding mapping function.

3. The source coding matrix data processing method based on the modified conjugate gradient algorithm according to claim 1, characterized in that, Perform a one-dimensional line search along the search direction to determine the optimal step size, including: the optimal step size of the one-dimensional line search. The step size is determined using a backtracking method, with an initial step size of 1 and a reduction factor of 0.5 each time, until the condition is met. , For loss function, For the first The encoding matrix of the next iteration For the first The search direction for the next iteration. is the Armijo condition constant.

4. The source coding matrix data processing method based on the modified conjugate gradient algorithm according to claim 1, characterized in that, In the fixed-point quantization operation performed on the sparsified coding matrix, the fixed-point quantization bit width is 8 bits, including 1 sign bit, 3 integer bits, and 4 fractional bits, and the quantization step size is... , and These are the historical maximum and minimum values ​​of all non-zero elements in the current encoding matrix during the first 100 iterations of training.

5. The source coding matrix data processing method based on the modified conjugate gradient algorithm according to claim 4, characterized in that, After each iteration of sparsification and quantization operations, a numerical stability check is performed on the encoding matrix. If any non-zero elements exceed the fixed-point representation range, a range recalibration mechanism is triggered, and the matrix is ​​recalculated. and And update the quantization parameters.

6. The source coding matrix data processing method based on the modified conjugate gradient algorithm according to claim 5, characterized in that, The sparse mask is updated in each iteration to an indicator matrix of the non-zero element positions in the current encoding matrix. Its dimension is the same as that of the encoding matrix, with the non-zero element positions corresponding to a value of 1 and the rest being 0.

7. The source coding matrix data processing method based on the modified conjugate gradient algorithm according to claim 6, characterized in that, The convergence condition is that the decrease in the loss function over three consecutive iterations is less than a preset convergence threshold.

Images