Method for quantizing large model and method for reasoning by using quantized large model

Abstract

The invention provides a method for quantizing a large model and a method for reasoning by using the quantized large model, and the quantization method comprises the following steps: carrying out data block granularity division on the large model, and then carrying out quantization according to a quantization parameter (encoding) of each block so as to obtain the quantized large model. According to the scheme, the reasoning precision of the quantized large model is effectively improved on the premise that the calculated amount is relatively small through quantization of the block granularity which is finer than the channel granularity.

Description

Technical Field

[0001] This application relates to the field of large model application technology, and in particular to a method for quantizing large models, a method for inference using quantized large models, electronic devices, chip systems, computer-readable storage media, and computer program products. Background Technology

[0002] For a model, during inference, the data to be processed is input into the model, and after calculation with the parameters in the model, the inference result is output. Taking a face recognition model as an example, an image can be input into the face recognition model. After the face recognition model processes the input image, it can recognize the face in the image. This recognition process is called the inference process. During model inference, the model's parameter matrix is ​​calculated with the input data. With the development of technology, the scale and complexity of models are increasing, leading to the emergence of large models. Large models have more refined and richer inference capabilities, but correspondingly, the parameter matrix of large models is more complex, and the amount of computation in the inference process increases dramatically. This results in excessively long latency in the computation process of large models.

[0003] To minimize the inference time of large models, various quantization methods have emerged. By quantizing large models, their size is reduced, thereby decreasing computational load and improving computational and inference efficiency. However, these traditional quantization methods also introduce new problems, namely, reduced inference accuracy of large models. Therefore, how to improve the inference accuracy of large models while ensuring a relatively small computational load is an urgent technical problem to be solved. Summary of the Invention

[0004] This application provides a method for quantizing large models, a method for inference using quantized large models, an electronic device, a chip system, a computer storage medium, and a computer program product, which can improve the inference accuracy of large models while ensuring that the computational load of large models is relatively small.

[0005] Firstly, a method for inference using a large model is provided, comprising: dividing each weight matrix of the large model into multiple data blocks, each weight matrix including multiple channels of data, each channel of data being divided into at least one block, and the data of at least one channel in each weight matrix being divided into at least two blocks; quantizing each block according to the quantization parameters (encoding) of each block in the multiple blocks to obtain multiple quantized blocks, wherein at least two blocks of at least one channel of data in at least one weight matrix of the large model have different encodings; and determining the quantized large model based on the multiple quantized blocks.

[0006] In the technical solution of this application, the inference accuracy of the large model after quantization is effectively improved with a relatively small amount of computation by quantizing at a finer block granularity.

[0007] In conjunction with the first aspect, some implementations of the first aspect divide each weight matrix of the large model into multiple blocks, including: dividing the data of each channel in each weight matrix of the large model into N blocks; at least two blocks of the N blocks of each channel have different encodings, where N is an integer greater than 1. In this implementation, the data of each channel is divided into multiple blocks. This division method is simple and easy to operate, and the number of blocks for each channel is the same, with each block having an equal size.

[0008] In conjunction with the first aspect, in some implementations of the first aspect, N is an integer greater than 2. The large model includes at least one first weight matrix. The first precision threshold of the network layer containing the first weight matrix is ​​less than or equal to a baseline preset precision threshold, which is a baseline value determined based on the precision thresholds of all network layers in the large model. At least one channel in the first weight matrix includes N blocks comprising adjacent first and second blocks, with the first and second blocks having the same encoding. In this implementation, the weight matrices of network layers with relatively low precision requirements are encoded sparsly, meaning that multiple adjacent blocks share the same encoding for simplification, a process that can be termed pruning.

[0009] In one example, the common encoding of the first block and the second block is either the original encoding of the first block, the original encoding of the second block, or the union of the original encodings of the first block and the original encoding of the second block. In this example, unifying the encodings of the two blocks to the encoding of one of the blocks, or taking the union of the two original blocks, is equivalent to determining which adjacent block encodings to merge into a new encoding by comparing the differences between the encodings of adjacent blocks. This can further improve the quantization quality, thereby improving the inference accuracy of the quantized large model.

[0010] In conjunction with the first aspect, in some implementations of the first aspect, the data in each weight matrix of the large model before quantization is floating-point data; the data in each weight matrix of the large model after quantization is fixed-point data. This implementation provides examples of common data type conversions. The partitioning is intended to convert the floating-point data of different blocks into the fixed-point data corresponding to the quantization parameters of that block, and the parameters of the large model after quantization will fall within a specific interval corresponding to that quantization parameter (i.e., which data interval within the entire data range of the fixed-point data).

[0011] In conjunction with the first aspect, in some implementations of the first aspect, the encoding of each block is determined based on the data distribution within that block. This implementation provides an example of how the encoding of each block is determined, given the data distribution of all data for the large model parameters contained within that block.

[0012] It should be understood that each block corresponds to a unique encoding, meaning that each large model parameter within that block will be quantized using that encoding. When different blocks correspond to different encodings, it's equivalent to large model parameters within different blocks being quantized using the encoding of the block containing that parameter.

[0013] Secondly, a method for inference using a quantized large model is provided. This method includes: dividing the data to be processed into multiple input data blocks according to a first preset partitioning rule; quantizing the multiple input blocks according to the quantization parameter (encoding) corresponding to each input block to obtain first input data; wherein at least two input blocks have different encodings; the first input data includes the quantized data of the multiple input blocks; the data to be processed includes at least one image frame or a feature vector of at least one image frame; processing the first input data using the quantized large model to obtain first output data; the quantized large model is any quantization method obtained in the first aspect; dividing the first output data into multiple output blocks according to a second preset partitioning rule; and dequantizing the multiple output blocks according to the encoding corresponding to each output block to obtain target output data.

[0014] It should be understood that the second aspect primarily utilizes the quantized large model obtained in the first aspect for inference; therefore, the technical effects are similar to those in the first aspect and will not be elaborated further. The second aspect mainly aims to adaptively process the input and output model data by utilizing the quantized large model for inference. However, it should be understood that the preset partitioning rules used for the input and output data do not need to be the same, nor do they need to be related to the partitioning rules of the large model. In other words, the partitioning rules used for quantizing the input data, the large model, and the output data are independent and do not interfere with each other.

[0015] It should also be understood that setting the first and second preset partitioning rules can achieve a one-time solution, eliminating the need to retest and set the partitioning rules every time new data arrives. Instead, it is only necessary to partition the blocks and find the encoding of each block based on the two preset partitioning rules.

[0016] In conjunction with the second aspect, in some implementations of the second aspect, the first preset partitioning rule includes a first reference size and a first sparse flag bit for partitioning all input blocks. The first sparse flag bit is used to indicate input blocks that share the same encoding with adjacent input blocks. Dividing the data to be processed into multiple input blocks according to the first preset partitioning rule, and quantizing the multiple input blocks according to the encoding corresponding to each input block, to obtain the first input data includes: dividing the data to be processed into multiple input blocks equally according to the first reference size; determining the encoding of each input block according to a first mapping table and the first sparse flag bit. The first mapping table is used to represent the encoding corresponding to each input block. In this implementation, partitioning input blocks based on a partitioning reference and determining the encoding of each input block based on the sparse flag bit and the mapping table is fast and convenient, eliminating the need to repeat the same operation each time and repeatedly try to determine the sparse flag bit.

[0017] In conjunction with the second aspect, in some implementations of the second aspect, the second preset partitioning rule includes a second reference size and a second sparse flag for partitioning all output blocks. The second sparse flag is used to indicate output blocks that share the same encoding with adjacent output blocks. The process involves partitioning the first output data into multiple output blocks according to the second preset partitioning rule, and then inverse-quantizing each output block according to its corresponding encoding to obtain the target output data. This includes: dividing the first output data into multiple output blocks according to the second reference size; and determining the encoding of each output block according to a second mapping table and the second sparse flag. The second mapping table represents the encoding corresponding to each output block. In this implementation, partitioning the output blocks based on the partitioning reference and determining the encoding of each output block based on the sparse flag and the mapping table is fast and convenient, eliminating the need to repeat the same operation and repeatedly try to determine the sparse flag.

[0018] In conjunction with the second aspect, in some implementations of the second aspect, the data to be processed and the target output data are floating-point data, while the first input data and the first output data are fixed-point data. This implementation provides examples of common data type conversions; the partitioning is intended to convert the floating-point data of different blocks into the fixed-point data corresponding to the quantization parameters of that block.

[0019] In conjunction with the second aspect, in some implementations of the second aspect, the encoding of each input block is determined based on the data distribution within that input block; and / or, the encoding of each output block is determined based on the data distribution within that output block. In this implementation, the encoding of each block is determined separately based on the data distribution of the input data and / or the output data.

[0020] Thirdly, a reasoning apparatus is provided, comprising a unit consisting of software and / or hardware for performing any one of the methods of the first and second aspects.

[0021] Fourthly, an electronic device is provided, the electronic device comprising: one or more processors, and a memory;

[0022] The memory is coupled to the one or more processors and is used to store computer program code, the computer program code including computer instructions, which the one or more processors invoke to enable the electronic device to implement either the first aspect or the second aspect.

[0023] Fifthly, a chip system is provided, the chip system being applied to an electronic device, the chip system including one or more processors, the one or more processors being configured to invoke computer instructions to enable the electronic device to implement any one of the methods of the first aspect and the second aspect.

[0024] Optionally, the chip system also includes a memory electrically connected to the processor.

[0025] Optionally, the chip system may also include a communication interface.

[0026] In a sixth aspect, a computer-readable storage medium is provided, the computer-readable storage medium including instructions that, when executed on an electronic device, enable the electronic device to implement any one of the methods of the first aspect and the second aspect.

[0027] In a seventh aspect, a computer program product is provided, comprising a computer program that, when executed by an electronic device, can implement any one of the methods of the first or second aspect. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of a large-scale model application scenario applicable to the embodiments of this application.

[0029] Figure 2 This is a schematic diagram of two traditional quantification methods.

[0030] Figure 3 This is a schematic diagram of a quantization scheme according to an embodiment of this application.

[0031] Figure 4 This is a schematic diagram of another quantization scheme according to an embodiment of this application.

[0032] Figure 5 This is a schematic flowchart illustrating a method for quantifying a large model according to an embodiment of this application.

[0033] Figure 6 This is a schematic flowchart illustrating a method for reasoning using a quantized large model, according to an embodiment of this application.

[0034] Figure 7 These are schematic diagrams illustrating the execution process of two types of inference using a quantified large model.

[0035] Figure 8 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application.

[0036] Figure 9 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0037] The embodiments of this application will now be described in conjunction with the accompanying drawings.

[0038] Figure 1 This is a schematic diagram illustrating a large-scale model application scenario applicable to an embodiment of this application. For example... Figure 1 As shown, a large model typically includes a training phase and an inference phase. During training, a large number of training samples are used to train the model. During inference, the model is used for inference tasks, such as classification or feature extraction. In the training phase, the training device acquires a large number of training samples to train the initial model of the large model, resulting in a trained large model. The trained large model possesses the corresponding capabilities. The prediction results are compared with the true results (labels) of the training samples to adjust the weights of the initial model, thus completing one training iteration. The specific process is not elaborated further.

[0039] Training devices, due to the need for extensive computation, often utilize large-scale data processing equipment with high computing power, such as cloud servers. Inference devices, on the other hand, can be terminal devices such as mobile phones, tablets, laptops, and personal computers.

[0040] Once a large model is trained, it will have corresponding capabilities, such as image recognition (identifying people or objects from images) or speech extraction (extracting text from audio signals). It may have one or more of these capabilities, which will not be listed one by one.

[0041] After training a large model, it needs to be deployed to an inference device. Since inference devices may have relatively weak computing power or different usage requirements, the deployment strategy may vary depending on the usage requirements of the inference device and its hardware and software resources (and / or computing power). For example, the trained large model may be simplified or subjected to hierarchical grouping or other processing before deployment to the inference device.

[0042] Once a large model is deployed on the inference device, it can be used for inference tasks. The inference device can collect or acquire inference objects and use the deployed large model to perform inference operations on the inference objects to obtain inference results. For example, for a face recognition task, the inference object can be a frame or an image. The deployed large model is used to perform inference operations on the image to obtain the face information (inference result) in that image. Other cases will not be listed one by one.

[0043] Because the computing power of inference devices is limited, the inference process (processing of input data using a large model) has a long inference time (computation time, inference latency).

[0044] With technological advancements, the application of large-scale models is expanding. For instance, in processing low-resolution images to high-resolution images on electronic devices like mobile phones and tablets (which can be understood as inference devices), diffusion models (such as ControlNet) are needed. These models can convert low-resolution images into high-resolution ones. Combined with... Figure 1 The previous approach involved quantizing the large model (in this case, a diffusion model) before deploying it on the inference device to reduce its computational cost. The quantized diffusion model was then deployed to the inference device, and inference could then be performed using this deployed model. This involved inputting a low-resolution image into the deployed diffusion model, and then performing calculations with the model's parameters to obtain and output a high-resolution image. This application addresses the problem of the quantization strategy used during quantization. Different quantization strategies yield different results, and existing quantization schemes cannot simultaneously meet the performance requirements of both computational cost and accuracy.

[0045] In other words, since the initial parameters of large models are mostly high-precision floating-point parameters, their scale is too large to be directly deployed on electronic devices with limited computing power, such as mobile phones and tablets. Therefore, floating-point parameters can be mapped to fixed-point parameters, such as 8 bits, 4 bits, 2 bits, 16 bits, etc., to complete quantization. The most common example is converting 32-bit large model parameters into 4-bit or 8-bit parameters. While quantization reduces the computational load of the model, it also reduces the processing precision. Many scenarios have high requirements for model precision. For example, with the diffusion model mentioned above, if the model processing precision is too low, the quality of the converted high-definition image cannot be guaranteed. On the other hand, if the model size is too large, the inference device cannot handle its excessive computational load, resulting in excessive latency, or even making it impossible to deploy the diffusion model on the inference device at all.

[0046] Common quantization schemes mainly include per-tensor quantization and per-channel quantization. The following section will combine... Figure 2 The two quantification schemes are explained.

[0047] Figure 2 This is a schematic diagram of two traditional quantification methods. Figure 2 In this context, x represents the input data and y represents the output data. The so-called reasoning using a large model means that the input data x is processed with the weight matrix W to obtain the output data y. Figure 2 In this context, W represents the weight matrix of the large model, also known as the parameter matrix. Quantization, or quantization, is the process of transforming the data in the original weight matrix (i.e., the weight matrix before quantization) to a specific range of fixed-point data. This range can be understood as a numerical interval; that is, the data is transformed into a sub-interval within the full range of fixed-point data. For example, when the converted fixed-point data is 8-bit, it means mapping the weight matrix to 8-bit integers between [0, 255). This allows a 32-bit floating-point weight matrix to be mapped to a sub-interval within [0, 255).

[0048] The quantized parameters can satisfy the formula: Quant = round(W / scale) + offset, where Quant represents the quantized parameter matrix, W represents the parameter matrix before quantization, scale represents the scaling factor, and offset represents the offset. A quantized parameter (encoding) includes both a scale and an offset. It can be seen that quantization can be understood to some extent as reducing the bit width of the weight matrix values ​​in a large model, thereby effectively reducing the computational load.

[0049] like Figure 2As shown in (a), per-tensor quantization allows each weight matrix or input / output feature map before quantization to have dimensions of (N, C, H, W), where N represents the batch size, C represents the number of channels (channel dimension), and H and W represent the height and width of the sample, respectively. During quantization, each corresponds to only one set of quantization parameters s, which is an encoding. This means that all values ​​within the weight matrix are converted to the same data range. Combining this with the formula above, all values ​​in the weight matrix share a single encoding. Figure 2 (s) shown in (a). Such a quantized model has low accuracy, making it unsuitable for some scenarios with high accuracy requirements.

[0050] To address this issue, per-channel quantization was developed. This method assigns C encodings (each encoding includes a scale and an offset) to each weight matrix or input / output feature map, with each channel corresponding to one encoding. This allows the quantized weight matrix to better adapt to the data distribution characteristics, effectively improving model accuracy compared to per-tensor quantization without increasing computational cost. Figure 2 As shown in (b), during per-channel quantization, each row of the weight matrix W (i.e., each channel) corresponds to a quantization parameter, with each row corresponding to one channel. The example here shows the encodings for the first four rows as s1, s2, s3, and s4, but it should be understood that the other rows not shown also each correspond to an encoding. It can be seen that... Figure 2 In (a), each row (each channel) uses the same encoding, which is s; Figure 2 In (b), the encoding used for each row (each channel) is not exactly the same; it is a more granular quantization of the channel dimensions.

[0051] In short, quantization strategies have emerged to reduce the computational cost of large models. The most common is per-tensor quantization, which quantizes all weight matrices of a large model using the same quantization parameters, mapping all parameters of the large model to the same data range. However, as the demands on large models increase, in addition to ensuring low computational cost, it is also desirable to maximize the inference accuracy of the quantized large model. Therefore, more refined per-channel quantization has emerged, which maps different channels of a parameter to slightly different data ranges. However, with further increases in accuracy requirements, such as in the task of converting low-resolution images to high-resolution images, the processing accuracy of large models after per-channel quantization still cannot meet the requirements. To solve this problem, the solution proposed in this application was developed.

[0052] This application proposes a novel quantization scheme for large-scale models, where each channel corresponds to multiple quantization parameters. This achieves a more refined quantization scheme compared to per-channel quantization, improving the inference accuracy of large-scale models while ensuring relatively low computational cost. It should be understood that the quantization scheme for large-scale models proposed in this application can be implemented in… Figure 1 During the deployment phase, the structure of the large model remains unchanged. Instead, before the inference phase, the large model undergoes more refined quantization using a better quantization strategy before deployment. During the inference phase, the input and output data are partitioned and quantized online, and the quantized large model is used for online inference. It should be understood that the quantized large model during the inference phase is only used for inference operations and will not be requantized.

[0053] Figure 3 This is a schematic diagram of a quantization scheme according to an embodiment of this application. Figure 3 The explanations regarding W, x, y, and s, as well as the formulas that can be used for quantization, can be found above and will not be repeated here. This application primarily employs a more refined granularity of quantization parameter settings. Figure 3 Taking the division of each row into 3 equal data blocks as an example, it's understood that each row could also be divided into 4 blocks, 2 blocks, or other numbers of blocks. Furthermore, it's also possible to avoid equal division; other cases will not be listed here. Each row of data corresponds to one channel, therefore it can also be referred to as data per channel or data per channel. For example... Figure 3 As shown, each channel corresponds to 3 blocks, and the quantization parameters for each block are calculated using quantization optimization techniques based on the data distribution range of that block. For example... Figure 3 As shown, s1-s3 represent different encodings. Figure 3 The encodings of the six blocks A1-A6 are s1, s2, s3, s4, s5 and s6 respectively, and the encodings of these six blocks are not completely the same.

[0054] like Figure 3 As shown, each channel (c in total) is divided into b ( Figure 3 Taking 3 as an example, for each tensor, when using per_channel quantization, there are c quantization parameters, resulting in c sets of quantized weight parameters. Figure 3 After the quantization scheme shown, b*c sets of quantization parameters need to be stored. Each quantization parameter needs to cover a data range of 1 / b of the original data. The min-max expectations of the quantization are more concentrated, resulting in higher expression accuracy. For example, Figure 3 The data ranges of the three blocks (A1-A3) in channel 1 (first row) are {[-8,8],[-4,4],[-2,2]}. If per-channel quantization is used, the quantization parameters for these three blocks are min = -8 and max = 8, resulting in a quantized data range of [-8,8]. However, in the block-wise quantization scheme of this application, the quantization parameters for A1 are min = -8 and max = 8; the quantization parameters for A2 are min = -4 and max = 4, improving the quantization precision by one bit compared to per-channel quantization; and the quantization parameters for A3 are min = -2 and max = 2, improving the quantization precision by two bits compared to per-channel quantization.

[0055] As can be seen, in this application, the quantization parameter (encoding) can include scale and offset. In addition, encoding can also be directly represented by min-max, that is, the numerical range can be directly represented by the minimum value min and the maximum value max.

[0056] Furthermore, if an outlier occurs within a block, causing a significant increase in the data range of that block, the weights covered by that expression interval (the block) will experience a decrease in precision after quantization. Traditional per-channel quantization affects the quantization precision of the entire channel, while the block-wise quantization scheme of this application only affects a specific block within a channel. This can reduce the impact of outliers to some extent, improving the stability and security of quantization.

[0057] The so-called block-level scaling scheme does not require all blocks to have different encodings; rather, it only requires that the encodings of multiple blocks within certain channels are not entirely identical, and can be set as needed. Compared to per-channel quantization, which is a channel-level encoding scheme, it does not require all channels to have different encodings, but only that the encodings of all channels are not entirely identical. It can be seen that the scheme in this application is a more fine-grained quantization scheme.

[0058] Figure 3 Taking the example of dividing each row into 3 blocks, it should be understood that such indiscriminate division into multiple blocks inevitably leads to the need to store more quantization parameters. Figure 2 and Figure 3 To a certain extent, per-channel quantization increases the number of quantization parameters that large models need to store by a factor of C compared to per-tensor, where C is the channel dimension. Furthermore, the block-wise quantization in this application increases the number of quantization parameters that large models need to store by a factor of b compared to per-channel quantization, where b is the number of blocks that need to be divided within each channel. This inevitably requires more storage space.

[0059] To further reduce computational load by using block-level quantization, this application also proposes a pruning scheme. For some network layers with relatively low accuracy requirements, the encoding of some blocks can be adjusted to be the same.

[0060] Figure 4 This is a schematic diagram of another quantization scheme according to an embodiment of this application. Figure 4 The explanations of W, x, y, and s, as well as the formulas that can be used for quantification, can be found above and will not be repeated here. Figure 4 and Figure 3 In comparison, the difference lies in, Figure 4 Further unification Figure 3 The encoding used by some blocks in the code, for example Figure 4 B1 in the text corresponds to Figure 3 The two blocks A1 and A2 both use the encoding s1; B6 corresponds to three blocks, all of which share the same encoding: s6. Figure 4 The example uses B1-B6 with different encodings s1-s6.

[0061] When unifying quantization parameters for multiple blocks, it is necessary to know which blocks need to be unified to which quantization parameter. At this time, it is necessary to identify the different blocks and identify the blocks that need to use the same encoding as other blocks.

[0062] In one implementation, a sparse flag is used to guide the merging of encodings. This sparse flag can be understood as a marker indicating the start and end positions of the sparsed encoding, or as indicating the starting and ending positions for the sparse processing. The electronic device can then query the corresponding encoding based on this sparse flag. It should also be understood that the start and end positions of the sparsed encoding can be the numbers of the start and end blocks, the row and column numbers of the first and last values ​​of the start and end blocks, or the row and column numbers of the Nth and Nth values ​​of the start and end blocks, where the value of N is determined by the size of each block.

[0063] Sparse flags can be quantization parameter numbers, meaning they number different quantization parameters. A sparse flag indicates that multiple adjacent blocks share the same quantization parameter number among the quantization parameter numbers assigned to each block. For example, each evenly divided block originally has an encoding, and assuming there are 6 evenly divided blocks with encoding numbers in the sequence {s1, s2, s3, s4, s5, s6}, after sparsification, it becomes the sequence {s1, s1, s2, s2, s2, s3}. This shows that adjacent blocks have the same encoding number. When storing sparse flags, the sequence {s1, s1, s2, s2, s2, s3} can be stored directly to indicate the quantization parameters of each block. Alternatively, a flag "f" can be set, allowing the sparse flags to be stored as {s1, f, s2, f, f, s3}. In this case, only three sets of quantization parameters and one flag sequence need to be stored. In other words, the sparse flag can be understood as an index table (mapping table) of the number of each partitioned block and the number of the encoding of that block.

[0064] The resulting blocks can be numbered using a sequence of accumulated positive integers. This can be done by accumulating the numbers for each block, or by numbering each block along a specific dimension.

[0065] Figure 5This is a schematic flowchart illustrating a method for quantizing a large model according to an embodiment of this application. The following is a further explanation... Figure 5 The steps shown will be explained. Figure 5 The method shown can be derived from Figure 1 The inference device shown is executed.

[0066] S501. Divide each weight matrix of the large model into multiple blocks.

[0067] Each weight matrix includes data from multiple channels, and the data from each channel is divided into at least one block. The data from at least one channel in each weight matrix is ​​divided into at least two blocks.

[0068] It should also be understood that there is no limitation on whether each row, column, or other arrangement of data corresponds to a channel. Furthermore, there is no limitation on the size of each block.

[0069] The proposed solution is a block-level quantization that is more refined than channel-level quantization, enabling large models to maintain relatively high inference accuracy even after quantization.

[0070] Figure 3 and Figure 4 All of these can serve as examples of the multiple blocks obtained in step S501, where, Figure 3 The example shows that the data in each channel is divided into multiple blocks. Figure 4 The example shows that some channels have data divided into multiple blocks, while other channels have data in only one block.

[0071] In one implementation, step S501 may include: dividing each weight matrix of the large model into multiple blocks, including: dividing the data of each channel in each weight matrix of the large model into N blocks; at least two blocks of each channel have different encodings, where N is an integer greater than 1. In this implementation, the data of each channel is divided into multiple blocks, a simple and easy-to-operate method where the number of blocks for each channel is the same, and the size of each block is also equal.

[0072] However, the above partitioning method also has some problems. For example, the data for each channel of some weight matrices may not be evenly divided into N blocks. In this case, padding is needed to fill the last block of each row with zeros to make it the same size as the other blocks, or the size of the last block of each row may be smaller than the other blocks. Another problem with this partitioning method is that it increases the number of quantization parameters that large models need to store by a factor of N compared to the channel-level (per_channel) partitioning method, which will increase the number of quantization parameters that large models need to store to some extent. However, it should be understood that this will not affect the computational load. To alleviate this problem, the sparsity scheme mentioned above can be used to merge some blocks.

[0073] In one implementation, N is an integer greater than 2. The large model includes at least one first weight matrix. The first precision threshold of the network layer containing the first weight matrix is ​​less than or equal to a baseline preset precision threshold, which is a baseline value determined based on the precision thresholds of all network layers in the large model. At least one channel in the first weight matrix includes N blocks comprising adjacent first and second blocks, with the first and second blocks having the same encoding. In this implementation, the weight matrices of network layers with relatively low precision requirements are encoded sparsly, meaning multiple adjacent blocks share the same encoding for simplification, a process known as pruning.

[0074] In one example, the data distribution differences between adjacent blocks corresponding to the aforementioned merged encodings are within a preset range. In this example, determining which block encodings to merge by comparing the data distribution differences between adjacent blocks, rather than merging randomly, can further improve quantization quality, thereby increasing the inference accuracy of the quantized large model.

[0075] In one example, the common encoding of the first block and the second block is either the original encoding of the first block, the original encoding of the second block, or the union of the original encodings of the first block and the original encoding of the second block. In this example, unifying the encodings of the two blocks to the encoding of one of the blocks, or taking the union of the two original blocks, is equivalent to determining which adjacent block encodings to merge into a new encoding by comparing the differences between the encodings of adjacent blocks. This can further improve the quantization quality, thereby improving the inference accuracy of the quantized large model.

[0076] In another example, a greedy algorithm is used to determine the encodings that need to be merged. However, it should be understood that other suitable algorithms can also be used to determine the encodings that need to be merged.

[0077] It should be understood that the above example illustrates a method of first dividing the encoding of a block equally and then merging the remaining blocks. This method is simple and easy to operate. However, in practical applications, other methods can be used for partitioning. For example, data from each channel can be sequentially acquired according to a preset block size sequence to complete the partitioning. Assuming the preset block size sequence is {2,4,8,4}, where the unit is the number of data points, then for each channel's data, 2 data points are taken as the data to be quantized for the first block, then 4 data points are taken as the data to be quantized for the second block, then 8 data points are taken as the data to be quantized for the third block, and then 4 data points are taken as the data to be quantized for the fourth block, thus completing the partitioning of the data for each channel. Then, quantization is performed based on the encoding corresponding to each block, completing the quantization of the data for each channel. However, this partitioning method requires storing the preset block size sequence and taking data one by one for partitioning, making it more complex than equal partitioning, but it has a wider range of applications. When all the values ​​in the preset block size sequence are the same, it is equivalent to the equal partitioning scheme.

[0078] It should also be understood that even if the blocks are divided in the above-mentioned uneven way, the encodings of some adjacent blocks can still be merged, which will not be elaborated further.

[0079] S502. Based on the quantization parameters (encoding) of each block in the multiple blocks, quantize each block to obtain multiple quantized blocks.

[0080] In a large model, at least two blocks of data from at least one channel in at least one weight matrix have different encodings.

[0081] In one implementation, encoding can include a scaling factor and an offset.

[0082] In another implementation, encoding is the range of data represented by minimum and maximum values.

[0083] Step S502 mainly involves quantization based on the encoding corresponding to each block, thereby achieving a customized quantization scheme at the block level. This customization is not for the sake of being different, but rather sets quantization parameters for each block separately. This will result in the quantization parameters of all blocks not being exactly the same, but they will generally not be completely different or exactly the same.

[0084] In one implementation, the encoding of each block is determined based on the data distribution within that block. This implementation provides an example of how the encoding of each block is determined: it is based on the data distribution of all data for the large model parameters contained within that block. For example, it can be calculated using quantization optimization techniques based on the data distribution range of that block; however, there are no restrictions on the specific quantization optimization techniques used.

[0085] It should be understood that each block corresponds to a unique encoding, meaning that each large model parameter within that block will be quantized using that encoding. When different blocks correspond to different encodings, it's equivalent to large model parameters within different blocks being quantized using the encoding of the block containing that parameter.

[0086] S503. Based on multiple quantized blocks, determine the quantized large model.

[0087] The quantized blocks can be restored to the weight matrices of the larger model, thus obtaining the quantized weight matrices and the quantized larger model.

[0088] In another implementation, the data in each weight matrix of the large model before quantization is floating-point data; the data in each weight matrix of the large model after quantization is fixed-point data. This implementation provides examples of common data type conversions; the partitioning is intended to convert the floating-point data of different blocks into the fixed-point data corresponding to the quantization parameters of that block.

[0089] Figure 5 The method shown primarily utilizes block-level quantization to achieve relatively high inference accuracy with relatively low computational cost for the quantized large model. The quantized large model can then be deployed to inference devices for inference computation.

[0090] Figure 6 This is a schematic flowchart illustrating a method for inference using a quantized large model, according to an embodiment of this application. The following is a description of... Figure 6 The steps shown will be explained. Figure 6 The method shown can be derived from Figure 1 The inference device shown is executed.

[0091] S601. Divide the data to be processed into multiple input data blocks according to the first preset partitioning rule, and quantize the multiple input blocks according to the quantization parameter (encoding) corresponding to each input block to obtain the first input data.

[0092] At least two of the multiple input blocks have different encodings. The first input data includes the quantized data of the multiple input blocks. The data to be processed includes at least one frame of image or the feature vector of at least one frame of image.

[0093] In one implementation, the first preset partitioning rule includes a first reference size and a first sparse flag bit for partitioning all input blocks. The first sparse flag bit is used to indicate input blocks that share the same encoding with adjacent input blocks. The process involves partitioning the data to be processed into multiple input blocks according to the first preset partitioning rule, and quantizing each input block according to its corresponding encoding to obtain the first input data. This includes: dividing the data to be processed into multiple input blocks according to the first reference size; and determining the encoding of each input block according to a first mapping table and the first sparse flag bit. The first mapping table represents the encoding corresponding to each input block. In this implementation, partitioning the input blocks based on the partitioning reference and determining the encoding of each input block based on the sparse flag bit and the mapping table is fast and convenient, eliminating the need to repeat the same operation and repeatedly try to determine the sparse flag bit.

[0094] It should be noted that, Figure 6 The principle of the sparse flags involved is similar to that during the quantization of large models. Therefore, the description of the sparse flags of large models above can be referenced here. Just replace the block above with the output block or input block here. No further explanation is needed.

[0095] S602. The first input data is processed using the quantized large model to obtain the first output data.

[0096] The quantized large model is Figures 3-5 The quantized large model obtained by the quantization method shown.

[0097] It should be understood that there is no need for the preset partitioning rule to be correlated with the preset partitioning rule corresponding to the quantized large model.

[0098] S603. Divide the first output data into multiple output blocks according to the second preset partitioning rule, and dequantize the multiple output blocks according to the encoding corresponding to each output block to obtain the target output data.

[0099] It should be understood that the preset partitioning rules used for input data and output data do not need to be the same, nor do they need to be related to the partitioning rules of the large model. In other words, the partitioning rules used for quantizing input data, large model, and output data are independent and do not interfere with each other.

[0100] It should also be understood that setting the first and second preset partitioning rules can achieve a one-time solution, eliminating the need to retest and set the partitioning rules every time new data arrives. Instead, it is only necessary to partition the blocks and find the encoding of each block based on the two preset partitioning rules.

[0101] In one implementation, the second preset partitioning rule includes a second reference size and a second sparse flag for partitioning all output blocks. The second sparse flag indicates output blocks that share the same encoding as adjacent output blocks. The first output data is partitioned into multiple output blocks according to the second preset partitioning rule, and the multiple output blocks are dequantized according to the encoding corresponding to each output block to obtain the target output data. This includes: dividing the first output data into multiple output blocks equally according to the second reference size; and determining the encoding of each output block according to a second mapping table and the second sparse flag. The second mapping table represents the encoding corresponding to each output block. In this implementation, partitioning output blocks based on a partitioning reference and determining the encoding of each output block based on the sparse flag and mapping table is fast and convenient, eliminating the need to repeat the same operation and repeatedly try to determine the sparse flag.

[0102] In one implementation, the data to be processed and the target output data are floating-point data, while the first input data and the first output data are fixed-point data. This implementation provides examples of common data type conversions; the partitioning is intended to convert the floating-point data of different blocks into the fixed-point data corresponding to the quantization parameters of that block.

[0103] In one implementation, the encoding of each input block is determined based on the data distribution within that input block; and / or, the encoding of each output block is determined based on the data distribution within that output block. In this implementation, the encoding of each block is determined separately based on the data distribution of the input data and / or the output data.

[0104] Figure 6 The method shown is mainly for large quantized models. It also divides and sparsifies the input data (data to be processed) and output data (first output data) of the large model, thereby enabling the large quantized model to perform inference operations better.

[0105] Figure 7 These are schematic diagrams illustrating the execution process of two types of inference using a quantified large model. Figure 7 (a) shows an example of the inference process of a large model after quantization using a traditional approach. Figure 7 Figure (b) shows an example of the inference process of the large model after quantization of the scheme in this application.

[0106] like Figure 7 As shown in (a), traditional solutions do not divide the data into blocks or customize the encoding for each block. Therefore, they do not require partitioning or sparsifying the input and output data. The model input (the data to be processed in the quantized large model) is simply converted from floating-point to fixed-point (FloatToTfN) before being input into the quantized large model (model.bin) for inference. The resulting model output (the data output from the quantized large model) is then converted from fixed-point to floating-point (TfNToFloat) to obtain the target output. It should be understood that the floating-point to fixed-point conversion can be called quantization, just like model quantization; therefore, the fixed-point to floating-point conversion can be called dequantization.

[0107] like Figure 7 As shown in (b), in this application's solution, the model input (data to be processed or input data) needs to be divided into multiple blocks at the block granularity. Then, the encoding corresponding to each block in the encodings is matched. This includes sparse flag bits, that is, the encoding to be used for each block is determined based on the sparse flag bits, thus obtaining multiple input blocks. After that, the multiple input blocks are quantized accordingly (conversion from floating-point to fixed-point, i.e., ...). Figure 7The first input data is obtained by using FloatToTfN as shown in (b). The first input data is then input into the quantized large model for inference operations to obtain the model output (an example of the first output data). The model output is then divided into block-level granularities, and sparsity processing and dequantization are performed. Figure 7 The target output data is obtained by using TfNToFloat as shown in (b).

[0108] Compare Figure 7 As can be seen from the two execution flows, this application's scheme, due to the addition of block-level partitioning and quantization operations, also performs partitioning and quantization or dequantization operations on the input and output data. Specific partitioning rules and details regarding sparse flags can be found in the relevant content on quantizing large models, and will not be elaborated further here.

[0109] The methods of the embodiments of this application have been described above with reference to the accompanying drawings. It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially, these steps are not necessarily executed in the order shown in the figures. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the steps or stages of other steps. The apparatus of the embodiments of this application will now be described with reference to the accompanying drawings.

[0110] Figure 8 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Figure 8 As shown, the electronic device 2000 includes a quantization unit 2001 and an inference unit 2002. The electronic device 2000 can be an electronic device such as a mobile phone, tablet computer, laptop computer, or personal computer.

[0111] The device 2000 can be used to execute any of the methods described above. For example, the quantization unit 2001 can be used to execute... Figure 5 The inference unit 2002 can be used to execute the steps shown. Figure 6 The steps are shown. For example, the quantization unit 2001 can also be used to perform... Figure 3 and Figure 4 The quantization process shown can be executed by the inference unit 2002. Figure 7 The reasoning process is shown in (b).

[0112] In one implementation, the device 2000 may further include a storage unit for storing relevant data. This storage unit may be integrated into any of the aforementioned units, or it may be a unit independent of all of the aforementioned units.

[0113] Figure 9 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Figure 9 As shown, the electronic device 3000 includes: at least one processor 3001 ( Figure 9 (Only one is shown) a processor, a memory 3002, and a computer program 3003 stored in the memory 3002 and executable on the at least one processor 3001, wherein the processor 3001 executes the computer program 3003 to implement the steps of any of the above methods.

[0114] Those skilled in the art will understand that Figure 9 This is merely an example of an electronic device and does not constitute a limitation on electronic devices. In practice, electronic devices may include more or fewer components than those shown in the illustration, or combinations of certain components, or different components. For example, they may also include input / output modules, network access modules, etc.

[0115] Processor 3001 may include one or more processing units, such as: CPU, application processor (AP), modem processor, GPU, image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or NPU, other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. Different processing units may be independent devices or integrated into one or more processors.

[0116] The controller can serve as the nerve center and command center of the electronic device 3000. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0117] In some embodiments, memory 3002 may be an internal storage unit of electronic device 3000, such as a hard disk or memory of electronic device 3000. In other embodiments, memory 3002 may be an external storage device of electronic device 3000, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on electronic device 3000. Optionally, memory 3002 may include both internal and external storage units of electronic device 3000. Memory 3002 is used to store operating system, application programs, bootloaders, data, and other programs, such as program code of computer programs. Memory 3002 may also be used to temporarily store data that has been output or will be output.

[0118] The processor 3001 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 3001 is a cache memory. This memory can store instructions or data that the processor 3001 has just used or is recurring. If the processor 3001 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 3001, and thus improves the efficiency of the system.

[0119] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0120] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0121] This application also provides an electronic device, which includes: one or more processors and a memory; the memory is coupled to one or more processors, and the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to enable the electronic device to perform the steps in any of the above methods.

[0122] This application also provides a chip system applied to an electronic device. The chip system includes one or more processors, which invoke computer instructions to cause the electronic device to perform the steps in any of the methods described above. Optionally, the chip system further includes a memory electrically connected to the processor. Optionally, the chip system may also include a communication interface.

[0123] This application also provides a computer-readable storage medium storing instructions that, when executed by an electronic device, can implement any of the methods described above. This computer-readable medium may include at least: any entity or device capable of carrying computer program code (instructions) to a photographic device / electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0124] This application also provides a computer program product, which includes a computer program that, when executed by an electronic device, can implement any of the above-described methods. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form.

[0125] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0126] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0127] In the embodiments provided in this application, it should be understood that the disclosed apparatus / devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0128] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0129] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0130] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0131] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0132] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0133] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for quantifying large models, characterized in that, include: Each weight matrix of the large model is divided into multiple data blocks. Each weight matrix includes data from multiple channels. The data from each channel is divided into at least one block. The data from at least one channel in each weight matrix is ​​divided into at least two blocks. Based on the quantization parameter encoding of each block in the plurality of blocks, each block is quantized to obtain a plurality of quantized blocks, wherein the encoding of at least two blocks of data in at least one channel of at least one weight matrix of the large model is different; Based on the multiple quantized blocks, determine the quantized large model.

2. The method according to claim 1, characterized in that, The step of dividing each weight matrix of the large model into multiple data blocks includes: The data of each channel in each weight matrix of the large model is divided into N blocks; at least two blocks of the N blocks of each channel have different encodings, where N is an integer greater than 1.

3. The method according to claim 2, characterized in that, N is an integer greater than 2. The large model includes at least one first weight matrix. The first precision threshold of the network layer in which the first weight matrix is ​​located is less than or equal to a baseline preset precision threshold. The baseline precision threshold is a baseline value determined based on the precision thresholds of all network layers in the large model. In the first weight matrix, at least one channel of N blocks includes an adjacent first block and a second block. The encoding of the first block and the second block is the same.

4. The method according to claim 3, characterized in that, The common encoding of the first block and the second block is the original encoding of the first block, the original encoding of the second block, or the union of the original encoding of the first block and the original encoding of the second block.

5. The method according to any one of claims 1 to 4, characterized in that, Before quantization, the data in each weight matrix of the large model are floating-point data; after quantization, the data in each weight matrix of the large model are fixed-point data.

6. The method according to any one of claims 1 to 4, characterized in that, The encoding of each block is determined based on the data distribution within that block.

7. A method for reasoning using a quantized large model, characterized in that, include: The data to be processed is divided into multiple input data blocks according to a first preset partitioning rule, and the multiple input blocks are quantized according to the quantization parameter encoding corresponding to each input block to obtain the first input data. At least two of the multiple input blocks have different encodings. The first input data includes the quantized data of the multiple input blocks. The data to be processed includes at least one frame of image or the feature vector of the at least one frame of image. The first input data is processed using the quantized large model to obtain the first output data, wherein the quantized large model is the quantized large model as described in any one of claims 1 to 5; The first output data is divided into multiple output blocks according to the second preset partitioning rule, and the multiple output blocks are dequantized according to the encoding corresponding to each output block to obtain the target output data.

8. The method according to claim 7, characterized in that, The first preset partitioning rule includes a first reference size and a first sparse flag bit for partitioning all input blocks. The first sparse flag bit is used to indicate input blocks that share the same encoding with adjacent input blocks. The step of dividing the data to be processed into multiple input blocks according to a first preset partitioning rule, and quantizing the multiple input blocks according to the encoding corresponding to each input block to obtain the first input data includes: The data to be processed is divided into the plurality of input blocks according to the first reference size; The encoding of each input block is determined based on the first mapping table and the first sparse flag bit. The first mapping table is used to represent the encoding corresponding to each input block.

9. The method according to claim 7 or 8, characterized in that, The second preset partitioning rule includes a second reference size and a second sparse flag bit for partitioning all output blocks. The second sparse flag bit is used to indicate output blocks that share the same encoding with adjacent output blocks. The step of dividing the first output data into multiple output blocks according to the second preset partitioning rule, and dequantizing the multiple output blocks according to the encoding corresponding to each output block to obtain the target output data includes: The first output data is divided into the plurality of output blocks according to the second reference size; The encoding of each output block is determined based on the second mapping table and the second sparse flag. The second mapping table is used to represent the encoding corresponding to each output block.

10. The method according to any one of claims 7 to 9, characterized in that, The data to be processed and the target output data are floating-point data, while the first input data and the first output data are fixed-point data.

11. The method according to any one of claims 7 to 10, characterized in that, The encoding of each input block is determined based on the data distribution in that input block; and / or, the encoding of each output block is determined based on the data distribution in that output block.

12. An electronic device, characterized in that, The electronic device includes: one or more processors, and memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the electronic device to perform the method as claimed in any one of claims 1 to 6, or 7 to 11.

13. A chip system, characterized in that, The chip system is applied to an electronic device, the chip system including one or more processors, the one or more processors being configured to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 6, or 7 to 11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed on an electronic device, cause the electronic device to perform the method as claimed in any one of claims 1 to 6, or 7 to 11.

15. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by an electronic device, causes the electronic device to perform the method as claimed in any one of claims 1 to 6, or 7 to 11.

Images