Model compression method and device, electronic equipment and storage medium

Abstract

The invention discloses a model compression method and device, electronic equipment and a storage medium, and belongs to the technical field of model processing. The method is applied to electronic equipment and comprises the steps that an original model weight is divided into N input sequences, the length of each input sequence is a preset length, and N is a positive integer; according to a preset transformation matrix, performing M times of compression transformation processing on each input sequence, and obtaining a direct current component of the output sequence after each time of compression transformation processing, a mean value of alternating current components of the output sequence after the first (M-1) times of compression transformation processing and an alternating current component of the output sequence after the Mth time of compression transformation processing; determining a compressed model weight corresponding to the original model weight according to the direct-current component of the output sequence after each compression transformation processing of each input sequence, the mean value of the alternating-current components of the output sequence after the first (M-1) times of compression transformation processing and the alternating-current component of the output sequence after the Mth time of compression transformation processing, and the model compression efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of model processing technology, and in particular to a model compression method, device, electronic device and storage medium. Background Art

[0002] Large Language Models (LLMs) have achieved excellent performance in natural language understanding and generation tasks and are used in various fields. However, these LLM models usually require a lot of memory due to the large number of model parameters.

[0003] When these large models are inferred in electronic devices, it is generally necessary to read the model weights from the file system into high-speed memory. For example, the processor chip of the electronic device can read these weights from the high-speed memory and perform calculations and inferences. However, due to the large number of model parameters, a lot of high-speed memory is required. The size and bandwidth of the high-speed memory often become the performance bottleneck of large model inference. In order to better adapt to model inference in electronic devices, the model is compressed, such as distillation, pruning, quantization and other operations, and the compressed model is used in electronic devices. However, the current model compression is post-training quantization (PTQ) or perceptual quantization (QAT) for specific downstream tasks. This process requires the model to be reconstructed or retrained, and there are problems such as complex model compression process and low efficiency. Summary of the invention

[0004] In order to solve the problems of related technologies and improve the efficiency of model compression in electronic devices, the embodiments of the present application provide a model compression method, device, electronic device and storage medium. The technical solution is as follows:

[0005] In one aspect, the present application provides a model compression method, applied to an electronic device, the method comprising:

[0006] Divide the original model weights into N input sequences, each of which has a preset length, where N is a positive integer;

[0007] According to the preset transformation matrix, each input sequence is subjected to M compression transformation processing, and the DC component of the output sequence after each compression transformation processing, the average value of the AC component of the output sequence after the first M-1 compression transformation processing, and the AC component of the output sequence after the Mth compression transformation processing are obtained; wherein the input sequence used for the next compression transformation processing is determined according to the AC component of the output sequence after the previous compression transformation processing; the DC component includes the first column elements in the output sequence after the compression transformation processing, the AC component includes other elements in the output sequence except the DC component, and the average value is calculated according to the element value of the AC component after each transformation; wherein M is a positive integer;

[0008] The compressed model weight corresponding to the original model weight is determined based on the DC component of the output sequence after each compression transformation of each input sequence, the average of the AC component of the output sequence after the first M-1 compression transformations, and the AC component of the output sequence after the Mth compression transformation.

[0009] In one aspect, the present application provides a model compression device, applied to an electronic device, the device comprising:

[0010] A sequence partitioning module is used to partition the original model weights into N input sequences, each of which has a preset length, where N is a positive integer;

[0011] The first transformation module is used to perform M compression transformation processes on each input sequence according to a preset transformation matrix, and obtain the DC component of the output sequence after each compression transformation process, the average value of the AC component of the output sequence after the first M-1 compression transformation processes, and the AC component of the output sequence after the M-th compression transformation process; wherein the input sequence used for the next compression transformation process is determined according to the AC component of the output sequence after the previous compression transformation process; the DC component includes the first column elements in the output sequence after the compression transformation process, the AC component includes other elements in the output sequence except the DC component, and the average value is calculated based on the element value of the AC component after each transformation; wherein M is a positive integer;

[0012] A model determination module is used to determine the compressed model weight corresponding to the original model weight based on the DC component of the output sequence after each compression transformation of each input sequence, the mean of the AC component of the output sequence after the first M-1 compression transformations, and the AC component of the output sequence after the Mth compression transformation.

[0013] In another aspect, the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program, and the computer program is executed by the processor to implement the model compression method as described in one aspect above.

[0014] In another aspect, the present application provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is executed by a processor to implement the model compression method as described in one aspect above.

[0015] On the other hand, an embodiment of the present application provides a computer program product. When the computer program product runs on a computer, it enables the computer to execute to implement the model compression method as described in one aspect above.

[0016] On the other hand, an embodiment of the present application provides an application publishing platform, which is used to publish a computer program product, wherein when the computer program product runs on a computer, the computer is enabled to execute to implement the model compression method as described in one aspect above.

[0017] The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least:

[0018] The electronic device divides the original model weight into N input sequences, each of which has a preset length, wherein N is a positive integer; performs M compression transformation processes on each input sequence according to a preset transformation matrix, and obtains the DC component of the output sequence after each compression transformation process, the mean of the AC component of the output sequence after the first M-1 compression transformation processes, and the AC component of the output sequence after the M-th compression transformation process; wherein the input sequence used for the subsequent compression transformation process is determined based on the AC component of the output sequence after the previous compression transformation process; the DC component includes the first column elements in the output sequence after the compression transformation process, the AC component includes other elements in the output sequence except the DC component, and the mean of the AC component is calculated based on the element value of the AC component after each transformation; wherein M is a positive integer; according to the DC component of the output sequence after each compression transformation process of each input sequence, the mean of the AC component of the output sequence after the first M-1 compression transformation processes, and the AC component of the output sequence after the M-th compression transformation process, the compressed model weight corresponding to the original model weight is determined. The terminal device of the present application directly performs M compression transformation processing on the divided input sequence through a preset transformation matrix, and determines the compressed model weights corresponding to the original model weights based on the DC component of the output sequence after each compression transformation processing of each input sequence, the average of the AC components of the output sequence after the first M-1 compression transformation processing, and the AC component of the output sequence after the Mth compression transformation processing. It does not require complex retraining and is not designed for specific downstream tasks. It has good adaptability and is more convenient for compressing various large models in electronic devices, thereby improving the efficiency of model compression. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 is a schematic structural diagram of an electronic device provided by an exemplary embodiment of the present application;

[0021] Figure 2 is a schematic diagram of an architecture of a model reasoning process of the electronic device described above in accordance with an exemplary embodiment of the present application;

[0022] Figure 3 It is a schematic diagram of a large model parameter compression rate distortion curve involved in an exemplary embodiment of the present application;

[0023] Figure 4is a method flow chart of a model compression method provided by an exemplary embodiment of the present application;

[0024] Figure 5 is a method flow chart of a model compression method provided by an exemplary embodiment of the present application;

[0025] Figure 6 is a schematic diagram of input and output of a transform domain function involved in an exemplary embodiment of the present application;

[0026] Figure 7 It is a schematic diagram of compression error after compressing the open source 13B model by the model compression method of the present application;

[0027] Figure 8 is a structural block diagram of a model compression device provided by an exemplary embodiment of the present application;

[0028] Fig. 9 It is a structural schematic diagram of another example of the model compression device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0030] The term "multiple" as used herein refers to two or more than two. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0031] The solution provided by the present application can be used in real-life scenarios involving the compression of machine learning models when people use electronic devices in their daily lives. To facilitate understanding, some terms and application scenarios involved in the embodiments of the present application are first briefly introduced below.

[0032] Large Language Model (LLM): refers to a deep learning model trained with a large amount of text data that can generate natural language text or understand the meaning of language text. Large language models can handle a variety of natural language tasks, such as text classification, question answering, and dialogue, and are an important path to artificial intelligence.

[0033] Post Training Quantization (PTQ): is a model compression method that quantizes the trained model weights and converts them into a lower precision representation, thereby reducing the model size and computational complexity. This method is used in the model deployment phase, which can significantly reduce storage and computing costs while maintaining model performance.

[0034] Quantization Aware Training (QAT): It is also a model compression method that adapts the model to quantization during training. This method introduces quantization noise or uses quantized activation functions during training, allowing the model to run inference at low precision while maintaining good performance. Quantization-aware training is often used to optimize the weights and activations of the model to achieve better performance after quantization.

[0035] With the development of science and technology, various electronic devices have appeared in people's daily lives. People can use electronic devices for work, entertainment, study, etc. Usually, most functions in electronic devices are based on various machine learning models, such as image recognition, speech recognition, language translation and other functions.

[0036] Please refer to Figure 1 , which shows a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application. Figure 1 The electronic device shown includes components such as a processor 110 , a memory 120 , a transceiver 130 , a display unit 140 , and an input unit 150 .

[0037] The processor 110 is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 120, and calling data stored in the memory 120, the processor 110 performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 110 may include one or more processing units; optionally, the processor 110 may integrate an application processor, which mainly processes operating devices, user interfaces, and application programs, etc. Of course, other processors may also be included, which are not listed here one by one.

[0038] The memory 120 can be used to store software programs and modules. The processor 110 executes various functional applications and data processing of the electronic device by running the software programs and modules stored in the memory 120. The memory 120 may mainly include a program storage area and a data storage area, wherein the program storage area may store operating devices, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device (such as audio data, phone book, etc.), etc. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0039] The transceiver 130 can provide wireless communication solutions for electronic devices, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. The transceiver 130 can be one or more devices integrating at least one communication processing module, for example, an antenna and a baseband processor are integrated into the transceiver 130, or an antenna and a modem processor are integrated into the transceiver 130, etc., which are not limited here.

[0040] The display unit 140 may be used to display information input by a user or information provided to a user and various menus of the electronic device. The display unit 140 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc., which is not limited here.

[0041] The input unit 150 can be used to receive input digital or character information, and to generate key signal input related to user settings and function control of the electronic device. Specifically, the input unit 150 can collect the user's operations on or near it, and drive the corresponding connection device according to a pre-set program. In addition, the input unit 150 may include a touch panel, which can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel, the input unit 150 may also include other input devices. Specifically, other input devices may include, but are not limited to, one or more of function keys (such as volume control keys, switch keys, etc.), trackballs, joysticks, etc.

[0042] Although not shown, the electronic device may further include a camera. Optionally, the camera may be located at the front or rear of the electronic device, which is not limited in the present embodiment.

[0043] It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0044] Optionally, the above-mentioned electronic devices may include but are not limited to wearable devices (such as smart bracelets, smart watches, smart glasses, etc.), mobile phones, tablet computers, laptops, smart glasses, smart watches, MP3 players (Moving Picture Experts Group Audio Layer III, Moving Picture Experts Compression Standard Audio Layer 3), MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Compression Standard Audio Layer 4) players, desktop computers, laptops, etc.

[0045] Usually, the above Figure 1 The electronic device shown needs to be equipped with a corresponding operating system and run based on the operating system. For example, the operating system of the electronic device can be an Android system, an iOS system, a Linux system, etc.

[0046] As large language models have achieved excellent performance in natural language understanding and generation tasks, they are used in various fields. Electronic device manufacturers can flexibly select different large language models and set them in electronic devices to provide the required functions. However, these LLM models usually require a lot of memory due to the large number of model parameters. When these large models are inferred in electronic devices, the model weights (weights) of the model generally need to be read from the file system to the high-speed memory. For example, the processor chip of the electronic device can read these weights from the high-speed memory and perform calculation reasoning.

[0047] Please refer to Figure 2 , which shows a schematic diagram of the architecture of a model reasoning process of the electronic device described above in an exemplary embodiment of the present application. Figure 2 As shown, in the electronic device, a flash memory 201, a high-speed memory system 202, and a processor 203 are included. Among them, the flash memory 201 can store the weight files (weight) of the trained large language model, and the processor 203 reads these weight files through the high-speed memory system 202 and performs calculation reasoning. However, there are many parameters in the large language model, which requires a lot of high-speed memory. The size and bandwidth of the high-speed memory often become the performance bottleneck of the large language model reasoning. For example, if the Llama 7B model is expected to achieve 20 tokens / s, each model parameter is represented by 1 Byte, which requires about 7GB of parameter memory and a high-speed memory bandwidth of about 140GB / s. This puts high demands on the memory, especially on electronic devices belonging to the end-side devices. It is difficult to achieve. Among them, the end-side device is relative to the server. For independent end-side devices that can be connected to the server, they can be regarded as the end-side devices of the server.

[0048] Optionally, the high-speed memory system 202 may be a double data rate synchronous dynamic random access memory (Double Data Rate, DDR).

[0049] Optionally, the processor 203 is an eXtensible Processing Unit (xPU) of an electronic device, and the xPU may be a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU) or other inference acceleration devices.

[0050] For the application of large models (for example, the large language model) on the above electronic devices, the existing solution is mainly model compression, such as model distillation, pruning, quantization and other operations, that is, using a model with lower bits, smaller models, and fewer parameters to run on the end side. At present, in large model applications, quantization is a necessary step to deploy large models to electronic devices. For example, the existing technologies for post-training quantization (PTQ, Post Training Quantization) or perceptual quantization (QAT, Quantization Aware Training) for specific downstream tasks can be implemented without losing basic accuracy. For example: use 4 bits to represent the model weights of a large model.

[0051] Please refer to Figure 3 , which shows a schematic diagram of a large model parameter compression rate distortion curve involved in an exemplary embodiment of the present application. Figure 3 As shown in the figure, the horizontal axis D represents the model accuracy loss, and the vertical axis R represents the average number of bits after the large model is compressed. For the compression process of the parameters of the large model, there is a nonlinear relationship curve between the quantized information (i.e., the average number of bits after the large model is compressed) R and the quantization loss (i.e., the model accuracy loss) D. In some of the above practices, different training methods can be used to find a model parameter representation method with a smaller average bit, i.e., a smaller R value, while keeping the model accuracy loss basically unchanged (D unchanged), thereby completing model compression.

[0052] Since the current research on lower-bit quantization solutions requires a lot of cloud retraining (such as QAT), and even the reconstruction of the model (for example, most existing 1-bit quantization methods require the reconstruction of the gradient constructor), there are still many training techniques to ensure the correct results, which makes the progress of lower-bit quantization solutions slow in practical applications. That is, this process requires the reconstruction or retraining of the model, and there are problems such as complex model compression process and low efficiency.

[0053] In order to solve the problems existing in the above-mentioned related technologies and improve the efficiency of model compression in electronic devices, the present application provides a model compression method, which can compress and transform the input sequence through a preset transformation matrix without reconstructing or retraining the model, thereby simplifying the model compression process and quickly obtaining the compressed model weights under the premise of limited loss of model accuracy.

[0054] Please refer to Figure 4 , which shows a method flow chart of a model compression method provided by an exemplary embodiment of the present application, and the model compression method can be applied to electronic devices. Figure 4 As shown, the model compression method may include the following steps:

[0055] Step 401, divide the original model weights into N input sequences, the length of each input sequence is a preset length, where N is a positive integer.

[0056] Optionally, when the electronic device compresses the original model, it first divides the original model weight into multiple blocks to be compressed, and then compresses each block to be compressed. That is, the process of dividing into multiple blocks to be compressed is the process of dividing the original model weight into N input sequences. For example, taking the input original model weight as 4-bit data as an example, the electronic device re-divides the input original 4-bit data into N input sequences, where N is a positive integer.

[0057] Optionally, the value of N is determined based on the data volume of the original model weight and the preset length. For example, the preset length can be 255, and the data volume of the original 4-bit data is K, then N = the integer part of the result of K / 255. The length of 255 here is exemplary, and the preset length in actual applications can be pre-set by the developer according to actual needs.

[0058] Optionally, through the above division, the original 4-bit data can be re-divided into N input sequences of preset lengths.

[0059] Step 402, according to the preset transformation matrix, each input sequence is compressed and transformed M times to obtain the DC component of the output sequence after each compression and transformation, the average of the AC component of the output sequence after the first M-1 compression and transformations, and the AC component of the output sequence after the Mth compression and transformation; wherein the input sequence used for the next compression and transformation is determined according to the AC component of the output sequence after the previous compression and transformation; the DC component includes the first column elements in the output sequence after the compression and transformation, the AC component includes other elements in the output sequence except the DC component, and the average of the AC component is calculated according to the element value of the AC component after each transformation; wherein M is a positive integer.

[0060] Optionally, the electronic device is also pre-set with a preset transformation matrix, which is used to perform compression transformation processing on each input sequence obtained by the above division. Through the preset transformation matrix, each input sequence is subjected to M compression transformation processing, where M ≥ 1 and is an integer. Optionally, the specific value of M can be flexibly set by the developer. For example, if M = 2, then each input sequence is subjected to 2 compression transformation processing according to the preset transformation matrix. If M = 4, then each input sequence is subjected to 4 compression transformation processing according to the preset transformation matrix.

[0061] Optionally, during each compression transformation process, when M is greater than or equal to 2, it is necessary to calculate the input sequence used in the next compression transformation process. In this solution, the input sequence used in the next compression transformation process is determined based on the AC component of the output sequence after the previous compression transformation process. That is, during the M-times compression transformation process, the AC component of the output sequence after each compression transformation process determines the input sequence to be used in the next compression transformation process. Of course, if M=1, then there is no need to perform the second compression transformation process, and there is no need to perform the corresponding determination of the input sequence used in the next compression transformation process.

[0062] Optionally, in the above compression transformation process, the DC component refers to the first column element in the output sequence after the compression transformation process, and the AC component refers to the other elements in the output sequence except the DC component. For example, under normal circumstances, after a compression transformation process is performed by a preset transformation matrix, the output sequence obtained is as follows: [x(0), ...x(253), x(254)]; the DC component of the output sequence can be [x(0)], and the AC component of the output sequence can be [x(1), ...x(253), x(254)].

[0063] Optionally, the electronic device can obtain the DC component of the output sequence after each compression transformation process and the AC component of the output sequence after each compression transformation process. In this solution, it is necessary to obtain the mean value of the AC component of the output sequence after the first M-1 compression transformation processes and the AC component of the output sequence after the Mth compression transformation process. The mean value of the AC component is calculated based on the element value of the AC component after each transformation.

[0064] Step 403, determines the compressed model weights corresponding to the original model weights based on the DC component of the output sequence after each compression transformation of each input sequence, the mean of the AC component of the output sequence after the first M-1 compression transformations, and the AC component of the output sequence after the Mth compression transformation.

[0065] Optionally, in this solution, the electronic device determines the model weight after the compression of the entire model, that is, the model weight after the model compression is completed, based on the DC component of the output sequence after each compression transformation of each input sequence obtained after the M compression transformation, the mean of the AC component of the output sequence after the first M-1 compression transformations, and the AC component of the output sequence after the Mth compression transformation, and stores it, thereby reducing the memory requirements of the electronic device. The compression of the original large model can be completed through the above process.

[0066] In summary, the electronic device divides the original model weight into N input sequences, the length of each input sequence is a preset length, wherein N is a positive integer; according to the preset transformation matrix, each input sequence is compressed and transformed M times, and the DC component of the output sequence after each compression and transformation, the mean of the AC component of the output sequence after the first M-1 compression and transformation, and the AC component of the output sequence after the M-th compression and transformation are obtained; wherein the input sequence used for the latter compression and transformation is determined according to the AC component of the output sequence after the previous compression and transformation; the DC component includes the first column elements in the output sequence after the compression and transformation, the AC component includes other elements in the output sequence except the DC component, and the mean of the AC component is calculated according to the element value of the AC component after each transformation; wherein M is a positive integer; according to the DC component of the output sequence after each compression and transformation of each input sequence, the mean of the AC component of the output sequence after the first M-1 compression and transformation, and the AC component of the output sequence after the M-th compression and transformation, the compressed model weight corresponding to the original model weight is determined. The terminal device of the present application directly performs M compression transformation processing on the divided input sequence through a preset transformation matrix, and determines the compressed model weights corresponding to the original model weights based on the DC component of the output sequence after each compression transformation processing of each input sequence, the average of the AC components of the output sequence after the first M-1 compression transformation processing, and the AC component of the output sequence after the Mth compression transformation processing. It does not require complex retraining and is not designed for specific downstream tasks. It has good adaptability and is more convenient for compressing various large models in electronic devices, thereby improving the efficiency of model compression.

[0067] Below, taking the compression transformation processing as an example, in which the transposed matrix of the preset transformation matrix is ​​multiplied by the transposed sequence of the input sequence to be compressed and transformed, M is at least 2. During the M-times compression transformation processing by the preset change matrix, each input sequence is processed as above in combination with the preset transformation matrix, and the corresponding DC component and AC component are obtained for subsequent calculation to complete the compression of the model.

[0068] Please refer to Figure 5 , which shows a method flow chart of a model compression method provided by an exemplary embodiment of the present application, and the model compression method can be applied to electronic devices. Figure 5 As shown, the model compression method may include the following steps:

[0069] Step 501, divide the original model weights into N input sequences, the length of each input sequence is a preset length, where N is a positive integer.

[0070] Optionally, the manner in which the electronic device divides the original model weights into N input sequences can refer to the description in the above step 401, which will not be repeated here.

[0071] Since the electronic device re-divides the original model weights, it is inevitable that there will be residual model weights in the division process. Still taking the above-mentioned preset length of 255 as an example, if the data volume K of the original 4-bit data is an integer multiple of 255, it can just be divided into K / 255 input sequences, and the subsequent steps can be performed on these input sequences. However, if the original model weights are divided into N input sequences, the remaining model weights form an input sequence less than the preset length, and the input sequence less than the preset length is not compressed. Transformation processing. For example: the original model weights are divided into 3 255 input sequences (i.e., 3 blocks to be compressed), and the remaining model weights are also divided in the same way, which can only be 120 in length, which is less than the preset length sequence. Then, for input sequences less than 255 in length, this scheme does not perform compression transformation processing.

[0072] Step 502: Perform a compression transformation process on each input sequence according to a preset transformation matrix to obtain an output sequence of the current compression transformation process.

[0073] Optionally, the compression transformation process is a process of multiplying the transposed matrix of the preset transformation matrix with the transposed sequence of the input sequence to be subjected to the compression transformation process. In this solution, the preset transformation matrix is ​​used to perform compression transformation on the input sequence, and the transformed output sequence can be obtained through the compression transformation process.

[0074] For example, taking an input sequence X among the N input sequences re-divided as above, the preset length is still 255, the input sequence X = [x(0), ...x(253), x(254)], the 255 elements in the input sequence X are used as a compression block, and each element has been quantized (for example, 4-bit quantization). By multiplying the transposed matrix of the preset transformation matrix with the transposed sequence of the input sequence X, a transformed output sequence can be obtained, and the transformed output sequence is the output sequence of the current compression transformation process.

[0075] Optionally, in this solution, the preset transformation matrix can be selected based on different transformation domain functions. For example, the transformation domain function can be various linear transformation domain functions or nonlinear transformation domain functions; among which, the representative of the linear transformation domain function can be the transformation function corresponding to the discrete Hadamard transform (Discrete Hadamard Transformer, DHT), the transformation function corresponding to the discrete Fourie transform (Discrete Fourie Transformer, DFT), the transformation function corresponding to the wavelet transform, etc. The representative of the nonlinear transformation domain function can be the sigmoid function, the ReLU function, etc.

[0076] Optionally, the above transform domain function can also be extended to an artificial intelligence (AI) network. By inputting the input sequence into the AI ​​network, an output sequence can be obtained. The output sequence can also contain DC components and AC components. Please refer to Figure 6 , which shows a schematic diagram of input and output of a transform domain function involved in an exemplary embodiment of the present application, such as Figure 6 As shown, a transform domain function module 601 is included, and various linear transform domain functions or nonlinear transform domain functions or AI networks can be used in the transform domain function module 601. By inputting the original input sequence into the transform domain function module 601 for processing, the corresponding DC component and AC component can be obtained.

[0077] Optionally, different preset transformation matrices may be set corresponding to the above-mentioned various transformation domain function settings, and the specific preset transformation matrix to be used may be determined according to the transformation domain function used by the compression algorithm. This solution takes the transformation domain function corresponding to the DHT transformation as an example, performs compression based on the DHT transformation, and selects the last 255 rows of the Hadamard matrix DHT(256) as the preset transformation matrix, and the size of the preset transformation matrix is ​​256*256. According to the preset transformation matrix, each of the above-mentioned input sequences is subjected to M compression transformation processes.

[0078] In one possible implementation, the electronic device of the present solution can interact with the developer, and the developer can manually select which transform domain function to determine the corresponding preset transform matrix. Alternatively, the preset transform matrix can also be a fixed one, which is pre-designed in the electronic device by the developer according to actual needs.

[0079] Optionally, still taking the transform domain function being the transform function corresponding to the DHT transform as an example, H represents the preset transform matrix, for the input sequence X=[x(0), ...x(253), x(254)] among the above N input sequences, a compression transform process can be performed as follows: by calculating H.transpose()*X.transpose(), Y=[y(0), ...y(254), y(255)] is obtained, and then the output sequence of the current compression transform process is obtained. Among them, H.transpose() represents the transposed matrix of the preset transform matrix H, and X.transpose() represents the transposed sequence of the input sequence X. Here, only a certain compression transform process is taken as an example, and the above operation transformation will be performed for each input sequence.

[0080] Step 503: Determine the next input sequence for the next compression transformation process according to the AC component of the output sequence of the current compression transformation process.

[0081] Optionally, in this solution, the input sequence used in the next compression transformation process is determined according to the AC component of the output sequence after the previous compression transformation process. That is, the AC component of the output sequence output after each compression transformation process is used to determine the next input sequence.

[0082] In one possible implementation, the electronic device determines the input sequence for the next compression transformation process in the following manner: the electronic device scales the AC component of the output sequence of the current compression transformation process according to the AC component of the output sequence of the current compression transformation process and a scaling factor parameter; obtains the minimum and maximum values ​​of the scaled AC component of the output sequence of the current compression transformation process, and calculates the mean value of the AC component of the output sequence after the current compression transformation process according to the minimum and maximum values; obtains the next input sequence for the next compression transformation process according to the scaled AC component of the output sequence of the current compression transformation process and the mean value of the AC component of the output sequence after the current compression transformation process.

[0083] The scaling factor parameter is used to scale the AC component of the output sequence after each compression transformation. The scaling factor parameter can be set by the developer based on experience. For example, scale represents the scaling factor parameter, scale = 4 is set, [y(1), ...y(254), y(255)] represents the AC component of the output sequence processed by the current compression transformation, and [y(1), ...y(254), y(255)] = [y(1), ...y(254), y(255)] / / scale is calculated, where " / / " means that the division result is rounded down. Thus, the AC component of the output sequence processed by the current compression transformation after scaling is obtained. The minimum value min and the maximum value max are selected from the sequence [y(1), ...y(254), y(255)] after scaling, and the mean value of the AC component of the output sequence processed by the current compression transformation is calculated based on the minimum value min and the maximum value max.

[0084] In one possible implementation, the electronic device calculates the mean of the AC component of the output sequence after the current compression transformation based on the minimum value min and the maximum value max as follows: calculate the average of the minimum value min and the maximum value max based on the minimum value min and the maximum value max, offset the average of the minimum value min and the maximum value max according to the offset factor, and use the average of the minimum value min and the maximum value max after the offset as the mean of the AC component of the output sequence after the current compression transformation.

[0085] The offset factor parameter is used to offset the average value of the minimum and maximum values ​​in the AC component of the output sequence after each compression transformation. For example, the minimum and maximum values ​​in the above [y(1), ...y(254), y(255)] are y(5) and y(154) respectively, and the average values ​​of the two are calculated as: mean_=(y(5)+y(154)) / / 2. The offset factor parameter is represented by mean_offset, then mean_=mean_+mean_offset is calculated to offset the average value mean_ of the minimum value min and the maximum value max, and the finally calculated mean_ is used as the mean value of the AC component of the output sequence after the current compression transformation.

[0086] Optionally, for the above situation, the electronic device performs the process of obtaining the next input sequence for the next compression transformation process based on the AC component of the output sequence of the current compression transformation process after scaling and the mean of the AC component of the output sequence of the current compression transformation process, and the mean of the AC component of the output sequence of the current compression transformation process is the offset mean. Optionally, the electronic device can determine the next input sequence for the next compression transformation process in the following manner: the next input sequence is represented by Y_p, Y_p = [y(1), ...y(254), y(255)]-mean_.

[0087] Among them, the scaling factor parameter and the offset factor parameter can be preset by the developer based on experience, and the specific values ​​are not limited. It should be noted that when the scaling factor parameter is 1 and the offset factor parameter is 0, it is equivalent to not substantially scaling the AC component of the output sequence of the current compression transformation process, and the unscaled AC component is still used. Therefore, in this case, it is equivalent to using the AC component of the output sequence of the current compression transformation process to determine the next input sequence for the next compression transformation process.

[0088] In one possible implementation, before executing step 503, the electronic device also adjusts the output sequence of the current compression transformation process according to the output sequence of the current compression transformation process and the proportional factor parameter; obtains the DC component and AC component of the adjusted output sequence of the current compression transformation process; in the method of determining the next input sequence for the next compression transformation process according to the AC component of the output sequence of the current compression transformation process in executing step 503, the next input sequence for the next compression transformation process is determined according to the adjusted AC component of the output sequence of the current compression transformation process.

[0089] Among them, the scale factor parameter is used to adjust the product result obtained each time. The scale factor parameter, like the scaling factor parameter, can also be preset by the developer based on experience. The scale factor parameter is represented by factor. After obtaining Y = [y(0), ...y(254), y(255)] through H.transpose()*X.transpose(), the scale factor parameter can be used to adjust the output sequence Y of the current compression transformation process. The adjustment method can be as follows: Y = Y / / factor. That is, let factor adjust all elements in the output sequence Y to obtain the adjusted [y(0), ...y(254), y(255)], and obtain the adjusted DC component and AC component of the output sequence of the current compression transformation process. Use the adjusted AC component of the output sequence of the current compression transformation process to perform the above step 503.

[0090] Step 504: Perform a compression transformation process on the next input sequence according to the preset transformation matrix to obtain an output sequence of the next compression transformation process, and so on, until M compression transformation processes are completed.

[0091] Optionally, after performing a compression transformation process, the next input sequence is obtained through the above steps 502 to 503, and the next input sequence is continuously subjected to a compression transformation process based on a preset transformation matrix until M compression transformation processes are completed. In other words, steps 502 to 504 are executed cyclically, from the first time to the Mth time, to complete M compression transformation processes on the N input sequences in the above step 501.

[0092] Here, taking M=2 as an example, the above steps 502 to 504 are illustrated. For an input sequence X obtained in the above step 501, the input sequence X=[x(0), ...x(253), x(254)], the preset transformation matrix H in this scheme is the last 255 rows of the selected Hadamard matrix DHT(256) as an example, and the above three preset factor parameters are: factor=6, scale=4, mean_offset=-2; for the first compression transformation process, the electronic device calculates H.transpose()*X.transpose() to obtain the output sequence Y=[y(0), ...y(254), y(255)] of the first compression transformation process; and uses the scale factor parameter to calculate the adjusted Y=Y / / dht_factor. The DC component and AC component in the adjusted Y are obtained respectively, the DC component is y(0), and the AC component is [y(1), ...y(254), y(255)].

[0093] The AC component [y(1), ...y(254), y(255)] of the output sequence of the first compression transformation is scaled by the scaling factor parameter scale, and [y(1), ...y(254), y(255)] = [y(1), ...y(254), y(255)] / / scale is calculated. For the minimum value min_ and the maximum value max_ of the output sequence [y(1), ...y(254), y(255)] after scaling, the average value mean_ = (min_ + max_) / / 2 is calculated. Mean_ is offset by the offset factor parameter mean_offset, and mean_ = mean_ + mean_offset is calculated. And based on the output sequence [y(1), ...y(254), y(255)] after scaling and the offset mean_, the next input sequence for the second compression transformation is calculated. Y_p represents the next input sequence of the second compression transformation process, Y_p = [y(1), ...y(254), y(255)]-mean_.

[0094] Perform a compression transformation process on Y_p again according to the preset transformation matrix, that is, the second compression transformation process, calculate H.transpose()*Y_p.transpose(), and adjust the result again using the above-mentioned scale factor parameter, scaling factor parameter and offset factor parameter.

[0095] Step 505, obtaining the DC component of the output sequence after each compression transformation process, the average of the AC component of the output sequence after the first M-1 compression transformation processes, and the AC component of the output sequence after the Mth compression transformation process.

[0096] Optionally, in the above-mentioned compression transformation process of executing M times, since the electronic device will perform the calculation of the content to be obtained in this step, the DC component of the output sequence obtained after each compression transformation process is the DC component after adjusting the output sequence of each compression transformation process by the proportional factor parameter. The method of obtaining the mean value of the AC component of the output sequence after the first M-1 compression transformation processes can be as follows: for the minimum and maximum values ​​in the AC component of the output sequence after each compression transformation process after scaling, after calculating the mean value of the AC component of the output sequence after each compression transformation process according to the minimum and maximum values, the mean value of the AC component of the output sequence after each compression transformation process is offset according to the offset factor, and the mean value of the AC component of the output sequence after the first M-1 compression transformation processes after the offset is obtained as the mean value of the AC component of the output sequence after the first M-1 compression transformation processes. For example, in the above-mentioned calculation process, the data to be obtained is recorded and obtained, and there is no need to calculate it separately again. The specific calculation process can refer to the above content and will not be repeated here.

[0097] Optionally, the method for obtaining the AC component of the output sequence after the Mth compression transformation processing can be as follows: after obtaining the AC component of the output sequence after the Mth compression transformation processing after scaling, trim each sequence element in the AC component of the output sequence after the Mth compression transformation processing after scaling to within the target interval, and obtain the AC component composed of each trimmed sequence element as the AC component of the output sequence after the Mth compression transformation.

[0098] That is to say, in addition to scaling the AC component of the output sequence after the last compression transformation according to the above-mentioned scaling factor parameter, this scheme also needs to adjust the AC component of the output sequence after the Mth compression transformation after scaling according to the scaling factor parameter, and trim each adjusted sequence element to within the target interval. After the electronic device executes the above process, the output sequence after the M-th compression transformation is represented by Z, Z = [z(0), ... z(254), z(255)]. In addition to calculating the adjusted DC component and AC component according to the above-mentioned scale factor parameter to obtain Z = Z / / factor, and using the DC component in the adjusted Z as the DC component of the output sequence after the M-th compression transformation, a scale operation is also performed, [z(1), ... z(254), z(255)] = [z(1), ... z(254), z(255)] / / scale, so as to scale the AC component of the output sequence after the M-th compression transformation, and clip the sequence [z(1), ... z(254), z(255)] after scaling to within the target interval. Taking the target interval range of [-2, 1] as an example, after clipping, the AC component composed of each sequence element is obtained as the AC component of the output sequence after the M-th compression transformation.

[0099] The process of trimming to [-2,1] can be as follows: use each value in [z(1), ...z(254), z(255)] to make a judgment. If the value of a sequence element is within the range of [-2,1], keep the sequence element and do not change it. If the value of a sequence element is not within the range of [-2,1], for example, if z(10) = 5, which is greater than 1, then z(10) = 1 and keep it at the original sequence position. If z(125) = -3, which is less than -2, then z(10) = -2 and keep it at the original sequence position. That is, trim each sequence element that is not within the target interval in the above manner.

[0100] Step 506, determines the compressed model weights corresponding to the original model weights based on the DC component of the output sequence after each compression transformation of each input sequence, the mean of the AC component of the output sequence after the first M-1 compression transformations, and the AC component of the output sequence after the Mth compression transformation.

[0101] Optionally, in the process of obtaining the DC component of the output sequence after each compression transformation, the mean of the AC component of the output sequence after the first M-1 compression transformations, and the AC component of the output sequence after the M-th compression transformation, the electronic device can quantize the acquired data in the following manner: quantize the DC component of the output sequence after each compression transformation according to the first quantization unit; quantize the mean of the AC component of the output sequence after the first M-1 compression transformations according to the second quantization unit; quantize the AC component of the output sequence after the M-th compression transformation according to the third quantization unit; wherein the third quantization unit is the target bit number of this model compression. The third quantization unit is the smallest of the first quantization unit, the second quantization unit, and the third quantization unit.

[0102] Optionally, take the quantization model where the weight of the original model is compressed to 4-bit data and the goal is to compress it to 2-bit as an example. The first quantization unit = 12-bit, the second quantization unit = 8-bit, and the third quantization unit = 2-bit quantization. That is, 12-bit quantization is used for the DC component of the output sequence after each compression transformation, 8-bit quantization is used for the mean of the AC component of the output sequence after the first M-1 compression transformations, and 2-bit quantization is used for the AC component of the output sequence after the Mth compression transformation.

[0103] In this scheme, taking M=2 as an example, the entire compression process is: divide the input original 4-bit data into multiple compression blocks (input sequences) of length 255, and execute the process from step 502 to step 504 for each input sequence. If the length of the input sequence cannot be divided by 255, the remaining sequence less than 255 in length will not be compressed.

[0104] The following is the design of the 2-bit compression algorithm based on DHT transformation proposed in this scheme. Take the input sequence X = [x(0), ...x(253), x(254)] as an example, the preset transformation matrix H is the last 255 rows of the selected Hadamard matrix DHT(256), and the set dht_factor = 6, scale = 4, mean_offset = -2. Execute the following process for X:

[0105] (1) Calculate H.transpose()*X.transpose() to obtain Y=[y(0),…y(254),y(255)].

[0106] Calculate Y=Y / / dht_factor.

[0107] (2) Save y(0) and use 12-bit quantization.

[0108] (3) Scale operation: [y(1),...y(254),y(255)]=[y(1),...y(254),y(255)] / / scale.

[0109] For the minimum value min_ and the maximum value max_ of the sequence [y(1), ...y(254), y(255)] after scaling, calculate the median mean_ = (min_ + max_) / / 2.

[0110] Let mean_=mean_+mean_offset.

[0111] Save mean_ and use 8-bit quantization.

[0112] (4) Calculate Y_p=[y(1),...y(254),y(255)]-mean_.

[0113] (5) Calculate H.transpose()*Y_p.transpose() to obtain Z=[z(0), ...z(254), z(255)]. Calculate Z=Z / / dht_factor.

[0114] (6) Save z(0) and use 12-bit quantization.

[0115] (7) Scale operation: [z(1),...z(254),z(255)]=[z(1),...z(254),z(255)] / / scale.

[0116] The values ​​of the sequence [z(1), ...z(254), z(255)] after scaling are clipped to the interval [-2, 1]. After clipping, each value is saved using 2 bits.

[0117] For M=3, the electronic device needs to perform the following process on X:

[0118] The first compression transformation process of X:

[0119] (1) Calculate H.transpose()*X.transpose() to obtain Y=[y(0),…y(254),y(255)].

[0120] Calculate Y=Y / / dht_factor.

[0121] (2) Save y(0) and use 12-bit quantization.

[0122] (3) Scale operation: [y(1),...y(254),y(255)]=[y(1),...y(254),y(255)] / / scale.

[0123] For the minimum value min_ and the maximum value max_ of the sequence [y(1), ...y(254), y(255)] after scaling, calculate the median mean_ = (min_ + max_) / / 2.

[0124] Let mean_=mean_+mean_offset.

[0125] Save mean_ and use 8-bit quantization.

[0126] The second compression transformation process of X:

[0127] (4) Calculate Y_p1=[y(1),...y(254),y(255)]-mean_.

[0128] (5) Calculate H.transpose()*Y_p1.transpose() to obtain Z1=[z1(0),…z1(254),z1(255)].

[0129] Calculate Z1=Z1 / / dht_factor.

[0130] (6) Save z1(0) and use 12-bit quantization.

[0131] (7) Scale operation: [z1(1), ...z1(254), z1(255)] = [z1(1), ...z1(254), z1(255)] / / scale.

[0132] For the minimum value min_ and the maximum value max_ of the sequence [z1(1), ...z1(254), z1(255)] after scaling, calculate the median mean_ = (min_ + max_) / / 2.

[0133] Let mean_=mean_+mean_offset.

[0134] Save mean_ and use 8-bit quantization.

[0135] The third compression transformation process of X:

[0136] (8) Calculate Y_p2 = [z1(1), ...z1(254), z1(255)] - mean_.

[0137] (9) Calculate H.transpose()*Y_p2.transpose() to obtain Z2=[z2(0),…z2(254),z2(255)].

[0138] Calculate Z2=Z2 / / dht_factor.

[0139] (10) Save z2(0) and use 12-bit quantization.

[0140] (11) Scale operation: [z2(1),...z2(254),z2(255)]=[z2(1),...z2(254),z2(255)] / / scale.

[0141] The values ​​of the sequence [z2(1), ...z2(254), z2(255)] after scaling are clipped to the interval [-2, 1]. After clipping, each value is saved using 2 bits.

[0142] It should be noted that the above processes are all examples for M=2 and M=3. In actual applications, M can be equal to more values, which can be determined by developers based on experience, and this solution does not limit this.

[0143] Optionally, for the solution provided by M=2 in this solution, an open source 13B model is used as an example to verify the compression algorithm provided above, and the compression error = abs (original parameter value - compressed / decompressed parameter value). Please refer to Figure 7 , which shows a schematic diagram of the compression error after the open source 13B model is compressed by the model compression method of the present application. Figure 7 As shown in the figure, the parameter values ​​are all expressed in 4 bits. From the figure, it can be seen that when compressing from 4 bits to 2 bits, the compression error is within 10%.

[0144] As for the loss of model accuracy, this case uses the Stories110M model as the model compression method of this solution with M=2. The test of the impact of model compression on model accuracy is as follows:

[0145] Table 1 Compression quantization accuracy test results

[0146]

[0147] When the electronic device is a mobile phone, actual test results on the mobile phone show that the impact of the compressed quantization model accuracy (PPL) of this scheme is less than 0.01.

[0148] In summary, the electronic device divides the original model weight into N input sequences, the length of each input sequence is a preset length, wherein N is a positive integer; according to the preset transformation matrix, each input sequence is compressed and transformed M times, and the DC component of the output sequence after each compression and transformation, the mean of the AC component of the output sequence after the first M-1 compression and transformation, and the AC component of the output sequence after the Mth compression and transformation are obtained; wherein the input sequence used for the latter compression and transformation is determined according to the AC component of the output sequence after the previous compression and transformation; the DC component includes the first column elements in the output sequence after the compression and transformation, the AC component includes other elements in the output sequence except the DC component, and the mean of the AC component is calculated according to the element value of the AC component after each transformation; wherein M is a positive integer; according to the DC component of the output sequence after each compression and transformation of each input sequence, the mean of the AC component of the output sequence after the first M-1 compression and transformation, and the AC component of the output sequence after the Mth compression and transformation, the compressed model weight corresponding to the original model weight is determined. The terminal device of the present application directly performs M compression transformation processing on the divided input sequence through a preset transformation matrix, and determines the compressed model weights corresponding to the original model weights based on the DC component of the output sequence after each compression transformation processing of each input sequence, the average of the AC components of the output sequence after the first M-1 compression transformation processing, and the AC component of the output sequence after the Mth compression transformation processing. It does not require complex retraining and is not designed for specific downstream tasks. It has good adaptability and is more convenient for compressing various large models in electronic devices, thereby improving the efficiency of model compression.

[0149] In addition, relative to the above Figure 2 For electronic devices in the present invention, this solution can not only reduce the size of the high-speed memory system occupied by the large model, but also reduce the bandwidth requirement of the high-speed memory system, thereby correspondingly reducing the power consumption of the high-speed memory system and improving the model compression efficiency of the electronic device.

[0150] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0151] Please refer to Figure 8 , which shows a structural block diagram of a model compression device provided by an exemplary embodiment of the present application. The model compression device 800 can be used in an electronic device. The terminal device can execute all or part of the steps performed by the electronic device in the method provided by each method embodiment. The model compression device 800 includes:

[0152] A sequence division module 801 is used to divide the original model weights into N input sequences, each of which has a preset length, where N is a positive integer;

[0153] The first transformation module 802 is used to perform M compression transformation processes on each input sequence according to a preset transformation matrix, and obtain the DC component of the output sequence after each compression transformation process, the average value of the AC component of the output sequence after the first M-1 compression transformation processes, and the AC component of the output sequence after the M-th compression transformation process; wherein the input sequence used for the next compression transformation process is determined according to the AC component of the output sequence after the previous compression transformation process; the DC component includes the first column elements in the output sequence after the compression transformation process, the AC component includes other elements in the output sequence except the DC component, and the average value of the AC component is calculated according to the element value of the AC component after each transformation; wherein M is a positive integer;

[0154] The model determination module 803 is used to determine the compressed model weight corresponding to the original model weight based on the DC component of the output sequence after each compression transformation processing of each input sequence, the mean of the AC component of the output sequence after the first M-1 compression transformation processing, and the AC component of the output sequence after the Mth compression transformation processing.

[0155] In summary, the electronic device divides the original model weight into N input sequences, the length of each input sequence is a preset length, wherein N is a positive integer; according to the preset transformation matrix, each input sequence is compressed and transformed M times, and the DC component of the output sequence after each compression and transformation, the mean of the AC component of the output sequence after the first M-1 compression and transformation, and the AC component of the output sequence after the Mth compression and transformation are obtained; wherein the input sequence used for the latter compression and transformation is determined according to the AC component of the output sequence after the previous compression and transformation; the DC component includes the first column elements in the output sequence after the compression and transformation, the AC component includes other elements in the output sequence except the DC component, and the mean of the AC component is calculated according to the element value of the AC component after each transformation; wherein M is a positive integer; according to the DC component of the output sequence after each compression and transformation of each input sequence, the mean of the AC component of the output sequence after the first M-1 compression and transformation, and the AC component of the output sequence after the Mth compression and transformation, the compressed model weight corresponding to the original model weight is determined. The terminal device of the present application directly performs M compression transformation processing on the divided input sequence through a preset transformation matrix, and determines the compressed model weights corresponding to the original model weights based on the DC component of the output sequence after each compression transformation processing of each input sequence, the average of the AC components of the output sequence after the first M-1 compression transformation processing, and the AC component of the output sequence after the Mth compression transformation processing. It does not require complex retraining and is not designed for specific downstream tasks. It has good adaptability and is more convenient for compressing various large models in electronic devices, thereby improving the efficiency of model compression.

[0156] Optionally, the compression transformation processing is a processing of multiplying the transposed matrix of the preset transformation matrix with the transposed sequence of the input sequence to be subjected to the compression transformation processing.

[0157] Optionally, the first transformation module 802 includes: a first acquisition unit, a first determination unit, and a second acquisition unit;

[0158] The first acquisition unit is used to perform a compression transformation process on each input sequence according to the preset transformation matrix to obtain an output sequence of the current compression transformation process;

[0159] The first determining unit is used to determine the next input sequence for the next compression transformation process according to the AC component of the output sequence of the current compression transformation process;

[0160] The second acquisition unit is used to perform a compression transformation process on the next input sequence according to the preset transformation matrix to obtain an output sequence for the next compression transformation process, and so on, until the M compression transformation processes are completed.

[0161] Optionally, the first determining unit is further configured to:

[0162] Scaling the AC component of the output sequence of the current compression transformation process according to the AC component of the output sequence of the current compression transformation process and the scaling factor parameter;

[0163] Obtaining the minimum value and the maximum value of the AC component of the output sequence after the current compression transformation processing after scaling, and calculating the mean value of the AC component of the output sequence after the current compression transformation processing according to the minimum value and the maximum value;

[0164] The next input sequence for the next compression transformation process is obtained according to the scaled AC component of the output sequence of the current compression transformation process and the mean value of the AC component of the output sequence after the current compression transformation process.

[0165] Optionally, before determining the next input sequence for the next compression transformation process according to the AC component of the output sequence of the current compression transformation process, the device further includes:

[0166] A first adjustment module, configured to adjust the output sequence of the current compression transformation process according to the output sequence of the current compression transformation process and a scale factor parameter;

[0167] A second acquisition module, used for acquiring the adjusted DC component and AC component of the output sequence of the current compression transformation process;

[0168] The first determining unit is further used to determine the next input sequence for the next compression transformation process according to the adjusted AC component of the output sequence of the current compression transformation process.

[0169] Optionally, the DC component of the output sequence after each compression transformation process is the DC component obtained by adjusting the output sequence after each compression transformation process by the proportional factor parameter.

[0170] Optionally, obtaining the mean value of the AC component of the output sequence after the first M-1 compression transformations, including:

[0171] For the minimum and maximum values ​​in the AC component of the output sequence after each compression transformation processing after scaling, after calculating the mean value of the AC component of the output sequence after each compression transformation processing based on the minimum and maximum values, the mean value of the AC component of the output sequence after each compression transformation processing is offset according to the offset factor, and the mean value of the AC component of the output sequence after the first M-1 compression transformation processing after the offset is obtained as the mean value of the AC component of the output sequence after the first M-1 compression transformation processing.

[0172] Optionally, obtaining the AC component of the output sequence after the M-th compression transformation processing includes:

[0173] After obtaining the AC component of the output sequence after the Mth compression transformation processing after scaling, each sequence element in the AC component of the output sequence after the Mth compression transformation processing after scaling is trimmed within the target interval, and the AC component composed of each trimmed sequence element is obtained as the AC component of the output sequence after the Mth compression transformation.

[0174] Optionally, the device further comprises:

[0175] A first quantization module, used for quantizing the DC component of the output sequence after each compression transformation according to the first quantization unit;

[0176] A second quantization module, used for quantizing the mean value of the AC component of the output sequence after the first M-1 compression transformations according to the second quantization unit;

[0177] The third quantization module is used to quantize the AC component of the output sequence after the M-th compression transformation according to the third quantization unit; wherein the third quantization unit is the target bit number of this model compression.

[0178] Optionally, if after the original model weights are divided into N input sequences, the remaining model weights form an input sequence smaller than the preset length, the device further includes:

[0179] The first processing module is used for not performing compression transformation processing on an input sequence that is shorter than the preset length.

[0180] Please refer to Fig. 9 , which is a structural diagram of another example of the model compression device provided in the embodiment of the present application. Among them, the model compression device 900 can be an electronic device that can implement the functions of the method provided in the embodiment of the present application. Among them, the model compression device 900 can be a chip system. In the embodiment of the present application, the chip system can be composed of a chip, or it can include a chip and other discrete devices.

[0181] In terms of hardware implementation, the above communication module may be a transceiver, which is integrated into the model compression device 900 to form a communication interface 903 .

[0182] The model compression device 900 includes at least one processor 901, which is used to implement or support the model compression device 900 to implement the functions of the electronic device in the method provided in the embodiment of the present application. Exemplarily, the processor 901 can execute the steps of dividing the original model weights into N input sequences; performing M compression transformation processes on each input sequence according to a preset transformation matrix, obtaining the DC component of the output sequence after each compression transformation process, the mean of the AC component of the output sequence after the first M-1 compression transformation processes, and the AC component of the output sequence after the Mth compression transformation process; according to the DC component of the output sequence after each compression transformation process of each input sequence, the mean of the AC component of the output sequence after the first M-1 compression transformation processes, and the AC component of the output sequence after the Mth compression transformation process, determining the compressed model weights corresponding to the original model weights, etc. Please refer to the detailed description in the method example for details, which will not be repeated here.

[0183] The model compression device 900 may also include at least one memory 902 for storing program instructions and / or data. The memory 902 is coupled to the processor 901. The coupling in the embodiment of the present application is an indirect coupling or communication connection between devices, units or modules, which may be electrical, mechanical or other forms, and is used for information exchange between devices, units or modules. The processor 901 may operate in conjunction with the memory 902. The processor 901 may execute program instructions stored in the memory 902. At least one of the at least one memory may be included in the processor.

[0184] The model compression device 900 may also include a communication interface 903, which is used to communicate with other devices through a transmission medium, so that the device in the model compression device 900 can communicate with other devices. Exemplarily, the other device may be a network side device. The processor 901 may use the communication interface 903 to send and receive data. The communication interface 903 may specifically be a transceiver.

[0185] The specific connection medium between the communication interface 903, the processor 901 and the memory 902 is not limited in the embodiment of the present application. Fig. 9 In the embodiment, the memory 902, the processor 901 and the communication interface 903 are connected via a bus 904. Fig. 9 The connections between other components are shown in bold lines, which are only for illustration and are not intended to be limiting. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0186] In the embodiment of the present application, the processor 901 can be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the methods, steps and logic block diagrams disclosed in the embodiment of the present application. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware processor to be executed, or can be executed by a combination of hardware and software modules in the processor.

[0187] In an embodiment of the present application, the memory 902 may be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), etc., or a volatile memory (volatile memory), such as a random access memory (RAM). The memory is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory in the embodiment of the present application may also be a circuit or any other device that can implement a storage function, for storing program instructions and / or data.

[0188] Optionally, an embodiment of the present application further provides an electronic device, which includes a processor and a memory, wherein the memory stores a computer program, and the computer program is executed by the processor to implement all or part of the steps performed by the electronic device in the model compression method of each of the above embodiments.

[0189] Optionally, an embodiment of the present application further provides a computer-readable medium storing a computer program, wherein the computer program is executed by a processor to implement all or part of the steps performed by an electronic device in the model compression method of each of the above embodiments.

[0190] Optionally, an embodiment of the present application further provides a computer program product, which, when executed on a computer, enables the computer to execute all or part of the steps executed by an electronic device to implement the model compression method of each of the above embodiments.

[0191] Optionally, an embodiment of the present application further provides an application publishing platform, which is used to publish a computer program product, wherein when the computer program product runs on a computer, the computer executes all or part of the steps performed by the electronic device to implement the model compression method of the above embodiments.

[0192] It should be noted that: when the device provided in the above embodiment performs the control of the electronic device, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0193] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0194] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0195] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A model compression method, characterized in that: Applied to electronic equipment, the method comprises: Divide the original model weights into N input sequences, each of which has a preset length, where N is a positive integer; According to the preset transformation matrix, each input sequence is subjected to M compression transformation processing, and the DC component of the output sequence after each compression transformation processing, the average value of the AC component of the output sequence after the first M-1 compression transformation processing, and the AC component of the output sequence after the Mth compression transformation processing are obtained; wherein the input sequence used for the next compression transformation processing is determined according to the AC component of the output sequence after the previous compression transformation processing; the DC component includes the first column elements in the output sequence after the compression transformation processing, the AC component includes other elements in the output sequence except the DC component, and the average value is calculated according to the element value of the AC component after each transformation; wherein M is a positive integer; The compressed model weight corresponding to the original model weight is determined based on the DC component of the output sequence after each compression transformation of each input sequence, the average of the AC component of the output sequence after the first M-1 compression transformations, and the AC component of the output sequence after the Mth compression transformation.

2. The method according to claim 1, characterized in that The compression transformation process is a process of multiplying the transposed matrix of the preset transformation matrix with the transposed sequence of the input sequence to be subjected to the compression transformation process.

3. The method according to claim 2, characterized in that When M is greater than or equal to 2, the step of performing M compression transformation processes on each input sequence according to a preset transformation matrix includes: According to the preset transformation matrix, each of the input sequences is subjected to a compression transformation process to obtain an output sequence of the current compression transformation process; Determining the next input sequence for the next compression transformation process according to the AC component of the output sequence of the current compression transformation process; According to the preset transformation matrix, a compression transformation process is performed on the next input sequence to obtain an output sequence of the next compression transformation process, and so on, until the M compression transformation processes are completed.

4. The method according to claim 3, characterized in that The step of determining the next input sequence for the next compression transformation process according to the AC component of the output sequence of the current compression transformation process comprises: Scaling the AC component of the output sequence of the current compression transformation process according to the AC component of the output sequence of the current compression transformation process and the scaling factor parameter; Obtaining the minimum value and the maximum value of the AC component of the output sequence after the current compression transformation processing after scaling, and calculating the mean value of the AC component of the output sequence after the current compression transformation processing according to the minimum value and the maximum value; The next input sequence for the next compression transformation process is obtained according to the scaled AC component of the output sequence of the current compression transformation process and the mean value of the AC component of the output sequence after the current compression transformation process.

5. The method according to claim 3, characterized in that: Before determining the next input sequence for the next compression transformation process according to the AC component of the output sequence of the current compression transformation process, the method further includes: Adjusting the output sequence of the current compression transformation process according to the output sequence of the current compression transformation process and the scale factor parameter; Acquire the adjusted DC component and AC component of the output sequence of the current compression transformation process; The step of determining the next input sequence for the next compression transformation process according to the AC component of the output sequence of the current compression transformation process comprises: The next input sequence for the next compression transformation process is determined according to the adjusted AC component of the output sequence of the current compression transformation process.

6. The method according to claim 5, characterized in that The DC component of the output sequence after each compression transformation process is the DC component obtained by adjusting the output sequence after each compression transformation process by the scale factor parameter.

7. The method according to claim 4, characterized in that Get the mean value of the AC component of the output sequence after the first M-1 compression transformations, including: For the minimum and maximum values ​​in the AC component of the output sequence after each compression transformation processing after scaling, after calculating the mean value of the AC component of the output sequence after each compression transformation processing based on the minimum and maximum values, the mean value of the AC component of the output sequence after each compression transformation processing is offset according to the offset factor, and the mean value of the AC component of the output sequence after the first M-1 compression transformation processing after the offset is obtained as the mean value of the AC component of the output sequence after the first M-1 compression transformation processing.

8. The method according to claim 4, characterized in that Obtain the AC component of the output sequence after the Mth compression transformation, including: After obtaining the AC component of the output sequence after the Mth compression transformation processing after scaling, each sequence element in the AC component of the output sequence after the Mth compression transformation processing after scaling is trimmed within the target interval, and the AC component composed of each trimmed sequence element is obtained as the AC component of the output sequence after the Mth compression transformation.

9. The method according to any one of claims 6 to 8, characterized in that: The method further comprises: quantizing the DC component of the output sequence after each compression transformation according to the first quantization unit; quantizing the mean value of the AC component of the output sequence after the first M-1 compression transformations according to the second quantization unit; The AC component of the output sequence after the M-th compression transformation is quantized according to the third quantization unit; wherein the third quantization unit is the target bit number of this model compression.

10. The method according to any one of claims 1 to 8, characterized in that: If after the original model weights are divided into N input sequences, the remaining model weights form an input sequence that is less than the preset length, the method further includes: No compression transformation is performed on input sequences that are shorter than the preset length.

11. A model compression device, characterized in that: Applied to electronic equipment, the device comprises: A sequence partitioning module is used to partition the original model weights into N input sequences, each of which has a preset length, where N is a positive integer; The first transformation module is used to perform M compression transformation processes on each input sequence according to a preset transformation matrix, and obtain the DC component of the output sequence after each compression transformation process, the average value of the AC component of the output sequence after the first M-1 compression transformation processes, and the AC component of the output sequence after the M-th compression transformation process; wherein the input sequence used for the next compression transformation process is determined according to the AC component of the output sequence after the previous compression transformation process; the DC component includes the first column elements in the output sequence after the compression transformation process, the AC component includes other elements in the output sequence except the DC component, and the average value is calculated based on the element value of the AC component after each transformation; wherein M is a positive integer; A model determination module is used to determine the compressed model weight corresponding to the original model weight based on the DC component of the output sequence after each compression transformation of each input sequence, the mean of the AC component of the output sequence after the first M-1 compression transformations, and the AC component of the output sequence after the Mth compression transformation.

12. An electronic device, characterized in that: The electronic device comprises a processor and a memory, wherein a computer program is stored in the memory, and the computer program is executed by the processor to implement the model compression method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the model compression method according to any one of claims 1 to 10.

Images