Quantized perception training method and related device

Abstract

The present application discloses a quantization-aware training method and related apparatus, comprising: obtaining a trained neural network model; calculating the second-order derivative corresponding to each network layer in the neural network model; constructing a linear programming problem for the neural network model, wherein the linear programming problem is used to determine the target bit position corresponding to each network layer in the neural network model; quantizing each network layer according to the target bit position corresponding to each network layer to obtain a mixed-precision quantization model; and performing quantization-aware training on the mixed-precision quantization model based on the neural network model. The present application embodiment is advantageously used to improve the optimization time used to determine the mixed-precision bit position setting, and is advantageously advantageous for better performance adaptation to hardware deployment requirements such as power.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a quantitative perception training method and related devices. Background Art

[0002] With the widespread application of deep learning algorithms in fields such as computer vision, speech processing, and autonomous driving, network model structures are being designed to become increasingly large to meet complex, high-precision, and multi-tasking scenarios. To meet the power consumption and computing resource constraints of deep learning algorithms during forward reasoning, deploying deep learning algorithms on edge devices or mobile platforms, which have strict constraints on power, latency, memory, and other resources, often requires a series of optimizations using compression algorithms.

[0003] Among them, the more common compression algorithms include quantization, pruning, distillation, etc. Quantization strategies generally include the following two types: Post-Training Quantization (PTQ) and Quantization-Aware Training (QAT).

[0004] Currently, the above quantization strategies are generally divided into single-precision quantization and mixed-precision quantization. The single-precision quantization process is relatively simple, but the hardware deployment requirements (such as power consumption) still exceed expectations. Mixed-precision quantization can optimize hardware deployment conditions such as power consumption and latency compared to single-precision quantization. However, using search or reinforcement learning methods in mixed-precision quantization will have a large time cost. Summary of the Invention

[0005] The embodiments of the present application provide a quantization-aware training method and related devices, which are beneficial to improving the optimization time used to determine the mixed precision bit settings, and are beneficial to better adapting to the performance requirements of hardware deployment such as power.

[0006] In a first aspect, an embodiment of the present application provides a quantization-aware training method, the method comprising:

[0007] Get the trained neural network model;

[0008] Constructing a linear programming problem for the neural network model, wherein the linear programming problem is used to determine the target bit corresponding to each network layer in the neural network model;

[0009] quantizing each network layer according to a target bit position corresponding to each network layer to obtain a mixed precision quantization model;

[0010] According to the neural network model, quantization-aware training is performed on the mixed precision quantization model.

[0011] In a second aspect, an embodiment of the present application provides a quantization perception training device, the device comprising: an acquisition unit, a construction unit, a quantization processing unit, and a quantization perception training unit, wherein:

[0012] The acquisition unit is used to acquire the trained neural network model;

[0013] The construction unit is configured to construct a linear programming problem for the neural network model, wherein the linear programming problem is used to determine a target bit corresponding to each network layer in the neural network model;

[0014] The quantization processing unit is used to perform quantization processing on each network layer according to the target bit corresponding to each network layer to obtain a mixed precision quantization model;

[0015] The quantization-aware training unit is used to perform quantization-aware training on the mixed-precision quantization model based on the neural network model.

[0016] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the program comprises instructions for executing the steps of any method of the first aspect of the embodiment of the present application.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the above-mentioned computer-readable storage medium stores a computer program for electronic data exchange, wherein the above-mentioned computer program enables a computer to execute part or all of the steps described in any method of the first aspect of the embodiment of the present application.

[0018] In a fifth aspect, embodiments of the present application provide a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, wherein the computer program is operable to cause a computer to execute some or all of the steps described in any method of the first aspect of the embodiments of the present application. The computer program product may be a software installation package.

[0019] It can be seen that in the embodiment of the present application, a trained neural network model is obtained; a linear programming problem is constructed for the neural network model, wherein the linear programming problem is used to determine the target bit corresponding to each network layer in the neural network model; each network layer is quantized according to the target bit corresponding to each network layer to obtain a mixed precision quantization model; and the mixed precision quantization model is trained for quantization perception according to the neural network model. In this way, the target bit corresponding to each network layer can be used for quantization processing, which is conducive to improving the optimization time used to determine the mixed precision bit setting; in actual deployment, the target bit actually corresponding to each network layer is constrained by the linear programming problem, which is conducive to better performance adaptation with hardware deployment requirements such as power, so that after the mixed precision quantization model is trained for quantization perception, it is conducive to improving the inference performance of the mixed precision quantization model. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 This is a schematic diagram of the structure of a communication system provided by an embodiment of the present application;

[0022] Figure 2 This is a flow chart of a quantitative perception training method provided by an embodiment of the present application;

[0023] Figure 3 This is a flow chart of a quantitative perception training method provided by an embodiment of the present application;

[0024] Figure 4 This is a flow chart of a method for determining a target loss function provided in an embodiment of the present application;

[0025] Figure 5 This is a flow chart of a quantitative perception training method provided by an embodiment of the present application;

[0026] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0027] Figure 7A This is a block diagram of the functional units of a quantitative perception training device provided in an embodiment of the present application;

[0028] Figure 7B This is a block diagram of the functional units of a quantitative perception training device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0030] The terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish between different objects, not to describe a particular order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0031] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0032] 1) The electronic device may be a portable electronic device that also includes other functions such as a personal digital assistant or music player function, such as a mobile phone, a tablet computer, a wearable electronic device with wireless communication function (such as a smart watch, smart glasses), a vehicle-mounted device, etc. Exemplary embodiments of portable electronic devices include but are not limited to portable electronic devices equipped with an IOS system, an Android system, a Microsoft system, or other operating systems. The above-mentioned portable electronic device may also be other portable electronic devices, such as a laptop computer (Laptop), etc. It should also be understood that in some other embodiments, the above-mentioned electronic device may not be a portable electronic device, but a desktop computer. In the embodiment of the present application, the above-mentioned electronic device may be a base station or a terminal device, and the terminal device may be a user equipment (UE).

[0033] 2) Single precision quantization, that is, each layer of the model uses the same bit for quantization operations.

[0034] 3) Mixed-precision quantization: each layer of the model uses a different bit number. Generally, 2 bits, 4 bits, 6 bits, 8 bits, etc. can be selected, but are not limited here.

[0035] 4) The quantization-aware training in the embodiment of the present application is to train the network during the quantization process so that the network parameters can better adapt to the information loss caused by quantization.

[0036] 5) The embodiments of the present application can generally support artificial intelligence (AI) neural network quantization for images and speech. The model described in the embodiments of the present application can refer to a neural network model.

[0037] Currently, the above quantization strategies are generally divided into single-precision quantization and mixed-precision quantization. The single-precision quantization process is relatively simple. Usually, the network is uniformly quantized to 8 bits to meet the edge deployment accuracy requirements, but the hardware deployment requirements (such as power consumption, etc.) still exceed expectations.

[0038] Mixed-precision quantization can optimize hardware deployment conditions such as power consumption and latency compared to single-precision quantization. However, human experience is usually not the optimal solution to meet hardware deployment conditions, and using search or reinforcement learning methods will have a high time cost.

[0039] Therefore, in response to the above problems, this application proposes a quantitative perception training method and related devices, which are described in detail below.

[0040] like Figure 1 , which is a structural diagram of the quantization-aware training model used in this application, the model may include: a training module 101, a data input module 102, a calculation module 103, a quantization module 104 and a quantization-aware training module 105.

[0041] Among them, the above-mentioned training module 101 can be connected to the data input module 102, the data input module 102 can be connected to the calculation module 103, the above-mentioned training module 101 can be connected to the calculation module 103, the calculation module 103 can be connected to the quantization module 104, the quantization module 104 and the quantization perception training module 105 are connected, and the target model is output through the quantization perception training module 105, that is, a mixed precision quantization model that meets the preset accuracy requirements.

[0042] In this application, training data may refer to data such as images or voice, and is not limited here.

[0043] Optionally, the electronic device can obtain a neural network model trained by the training module 101; the optional bits corresponding to the training data (which can be 4 bits, 6 bits, 8 bits, etc.) can be input into the calculation module 103 through the data input module 102, and the training data can be input through the data input module 102, and the derivatives corresponding to each network layer in the neural network model can be calculated through the calculation module 103 according to the optional bits; further, the power list and other data required for the forward reasoning calculation of each operator in the neural network model can be input into the calculation module 103 through the data input module 102, and a linear programming problem can be constructed for the neural network model according to the derivatives corresponding to each network layer through the calculation module 103, and based on the linear programming problem, the target bits corresponding to each network layer in the neural network model can be further calculated or determined, wherein each network layer can correspond to different target bits.

[0044] Furthermore, the quantization module 104 quantizes each network layer according to the target bit corresponding to each network layer to obtain a mixed precision quantization model; finally, the quantization perception training module 105 can perform quantization perception training on the mixed precision quantization model according to the neural network model; and the target model is output through the quantization perception training module 105, that is, the mixed precision quantization model that meets the preset accuracy requirements.

[0045] It can be seen that in this example, a trained neural network model is obtained; a linear programming problem is constructed for the neural network model, wherein the linear programming problem is used to determine the target bit corresponding to each network layer in the neural network model; each network layer is quantized according to the target bit corresponding to each network layer to obtain a mixed precision quantization model; and the mixed precision quantization model is subjected to quantization-aware training according to the neural network model. In this way, quantization processing can be performed based on the target bit corresponding to each network layer, which is conducive to improving the optimization time used to determine the mixed precision bit setting; in actual deployment, the linear programming problem is used to constrain the actual target bit corresponding to each network layer, which is conducive to better performance adaptation with hardware deployment requirements such as power, so that after the mixed precision quantization model is subjected to quantization-aware training, the inference performance of the mixed precision quantization model is improved.

[0046] It should be noted that in this application, multiple may refer to two or more, which will not be repeated later.

[0047] See also Figure 2 , Figure 2 This is a flow chart of a quantization perception training method provided in an embodiment of the present application. As shown in the figure, this quantization perception training method includes the following operations.

[0048] S201. Obtain a trained neural network model.

[0049] The neural network model may be a multi-bit floating point precision neural network model or a floating point precision neural network model. For example, it may be 4-bit, 8-bit, 16-bit, or 32-bit. Generally, the trained neural network model is 32-bit.

[0050] In this application, model quantization is a technique for converting floating-point calculations into low-bit fixed-point calculations. The electronic device may provide optional bits of training data or training parameters, for example, 4 bits, 6 bits, 8 bits, etc.

[0051] The training data may refer to image data or voice data that need to be trained, etc.

[0052] Before obtaining the trained neural network model, the neural network model may be trained in advance based on training data.

[0053] Among them, the above-mentioned neural network model can be a multi-layer network structure.

[0054] S202. Construct a linear programming problem for the neural network model, wherein the linear programming problem is used to determine the target bit corresponding to each network layer in the neural network model.

[0055] The target bit position may be different from the optional bit position of the given training data or training parameters.

[0056] The target bit number can be 2 bits, 4 bits, 6 bits, 8 bits, etc., and is not limited here. The target bit number is an integer number of bits. The target bit number can refer to bits corresponding to different network layers, and the target bit numbers corresponding to different network layers can be different. There is no limit on the combination of target bits corresponding to each network layer.

[0057] The linear programming problem can be used to determine the target bit number for each network layer. The constraints of the linear programming problem can be set by the user or by the system default and are not limited here. For example, the constraints of the linear programming problem can be determined based on hardware deployment requirements (such as power consumption), and the number or type of constraints is not limited.

[0058] S203 : quantize each network layer according to the target bit corresponding to each network layer to obtain a mixed precision quantization model.

[0059] The mixed precision quantization model is different from the neural network model. The mixed precision model can be a multi-layer network structure.

[0060] In a specific implementation, the electronic device can combine the target bits corresponding to each layer of the network to obtain the optimal bit, and quantize each network layer based on the optimal bit to obtain a mixed precision quantization model.

[0061] The quantization process may refer to a process of converting floating-point calculations into low-bit fixed-point calculations.

[0062] The optimal bit position may be determined based on the disturbance information corresponding to each network layer.

[0063] S204: Perform quantization-aware training on the mixed-precision quantization model according to the neural network model.

[0064] The electronic device can use a trained neural network model as a teacher model and a mixed-precision quantization model as a student model. This "teaching" approach accelerates the convergence of the mixed-precision quantization model, and iteratively trains the mixed-precision quantization model using quantization-aware algorithms until it reaches the preset accuracy requirements.

[0065] It can be seen that the quantization-aware training method described in the embodiment of the present application obtains a trained neural network model; constructs a linear programming problem for the neural network model, wherein the linear programming problem is used to determine the target bit corresponding to each network layer in the neural network model; performs quantization processing on each network layer according to the target bit corresponding to each network layer to obtain a mixed precision quantization model; and performs quantization-aware training on the mixed precision quantization model according to the neural network model. In this way, quantization processing can be performed based on the target bit corresponding to each network layer, which is conducive to improving the optimization time used to determine the mixed precision bit setting; in actual deployment, the linear programming problem is used to constrain the actual target bit corresponding to each network layer, which is conducive to better performance adaptation with hardware deployment requirements such as power, so that after the mixed precision quantization model is quantized and trained, it is conducive to improving the inference performance of the mixed precision quantization model.

[0066] In one possible example, the method for constructing a linear programming problem for the neural network model may include the following steps: performing derivative processing on each network layer in the neural network model to determine the derivative corresponding to each network layer; and constructing a linear programming problem for the neural network model based on the derivative corresponding to each network layer.

[0067] The above-mentioned derivation processing can be set by the user or by the system default, which is not limited here; for example, a second derivative can be performed on each network layer, and the second-order derivative corresponding to each network layer can be determined.

[0068] The derivative corresponding to each network layer obtained by the above derivation process can be used to determine the perturbation change rate of each network layer relative to the input data.

[0069] In the specific implementation, a part of the training data can be used to perform forward propagation calculation and back propagation calculation on the neural network model, which can be done by Figure 1 The calculation module shown is implemented to calculate the second-order derivative of the loss function (loss) of each network layer containing trainable parameters in the neural network model with respect to the weight parameters.

[0070] It can be seen that in this example, adopting this example is conducive to reducing the impact of the output of the forward propagation calculation, thereby helping to ensure training accuracy.

[0071] In addition, in this example, for example, if a second-order derivative is performed on each network layer, if the second-order derivative of a certain network layer is larger (for example, the second-order derivative is greater than or equal to a preset threshold, which can be set by the user or by the system default and is not limited here), it means that the rate of change of its perturbation to the input data is larger. In general, the network layer can use a larger first bit (for example, 8 bits) to represent the weight parameter, so that the training accuracy of the subsequent mixed-precision quantization model can be guaranteed. On the contrary, if the second-order derivative of a certain network layer is smaller (for example, the second-order derivative is less than the preset threshold), it means that the rate of change of its perturbation to the input data is smaller. In general, the network layer can use a smaller second bit (for example, 4 bits) to represent the weight parameter. In this way, by determining the derivative corresponding to each network layer and constructing a linear programming problem based on the derivative, it is beneficial to better determine the number of quantization bits of each layer of the mixed-precision quantization model, that is, the target number of bits, which is beneficial to better save optimization cost and time.

[0072] In a possible example, constructing a linear programming problem for the neural network model based on the derivatives corresponding to each network layer, the above method may include the following steps: determining the disturbance information corresponding to each network layer based on the derivatives corresponding to each network layer; obtaining the mixed bits of the training parameters corresponding to the neural network model; and constructing the linear programming problem based on the mixed bits and the disturbance information corresponding to each network layer.

[0073] The perturbation information corresponding to each network layer may refer to the degree of information change during the process of parameter quantization from floating point to fixed point. In the subsequent quantization process of each network layer, the lower the corresponding bit position, the greater the information change.

[0074] The mixed bits mentioned above may refer to optional bits corresponding to the training data (which may be 4 bits, 6 bits, 8 bits, etc.).

[0075] It can be seen that in this example, the derivative information corresponding to each network layer can be used to construct the linear programming problem to further obtain the target bit position, which is conducive to improving the optimization time used to determine the mixed precision bit position setting.

[0076] In a possible example, the linear programming problem is constructed based on the mixed bit position and the disturbance information corresponding to each network layer. The above method may include the following steps: generating an objective function based on the mixed bit position and the disturbance information corresponding to each network layer; obtaining the power consumed by the operator corresponding to each network layer during the first forward reasoning calculation; using the power corresponding to each network layer as a constraint condition, and constructing a linear programming problem based on the objective function.

[0077] Among them, the above objective function can be expressed as: Among them, f is the disturbance information of the mixed bit at each network layer, b i It is a type of mixed bit, which can be 4bit, 6bit, etc.

[0078] Since different operators quantize the same bit information differently, the above objective function also includes the derivative corresponding to each network layer, for example, the second-order derivative.

[0079] The first forward inference calculation mentioned above may also be the first forward propagation calculation, which may refer to directly using a model to perform inference based on an input and being able to obtain the power consumed by each operator at the hardware level.

[0080] In one possible example, the method for determining the target bit corresponding to each network layer in the neural network model may include the following steps: determining the sum of the disturbance information corresponding to each network layer to obtain comprehensive disturbance information; and determining, according to the linear programming problem, the target bit corresponding to each network layer when the comprehensive disturbance information is minimized.

[0081] The above linear programming problem can refer to the problem of total network power constraint. Under the condition of minimum comprehensive disturbance information, the target bit position of each network layer is obtained or determined.

[0082] It should be noted that in this application, power consumption is used as a constraint condition, but in actual deployment, the constraint condition may also be other hardware constraints in addition to power, which is not limited here.

[0083] It can be seen that in this example, the derivative information corresponding to each network layer can be used and combined with the derivative information to construct a linear programming problem to obtain the target bit position corresponding to each layer of the network, which is beneficial to improving the optimization time used to determine the mixed precision bit position setting.

[0084] It should be noted that in this application, the types and number of constraints are not limited. Therefore, in actual deployment, it is conducive to better performance adaptation with hardware deployment requirements such as power.

[0085] In a possible example, the mixed precision quantization model is subjected to quantization-aware training based on the neural network model. The method may include the following steps: inputting the training data into the neural network model and the mixed precision quantization model respectively, so as to perform a second forward inference calculation on the neural network model and the mixed precision quantization model respectively to obtain a first model and a second model; determining the target loss function of the mixed precision quantization model; and updating the gradient and weight corresponding to the mixed precision quantization model according to the target loss function.

[0086] Among them, the above-mentioned second forward reasoning calculation can also be a second forward propagation calculation. Different from the above-mentioned first forward reasoning calculation, the second forward reasoning calculation can refer to the training data in the mixed precision quantization model training process, which is the training data input into the network and the output is calculated layer by layer.

[0087] The target loss function mentioned above can refer to performing a second forward inference calculation on both the neural network model and the mixed-precision quantization model, respectively, to impart the mixed-precision quantization model with the reasoning capabilities of the neural network model. In a specific implementation, a knowledge distillation method can be introduced, using the neural network model as the teacher model and the mixed-precision quantization model as the student model. This "teaching" approach can be used to accelerate the model's convergence, allowing the quantized mixed-precision quantization model to possess the reasoning capabilities of the neural network model.

[0088] As can be seen, in this example, the knowledge distillation method is introduced to perform backpropagation calculations on the mixed-precision quantization model through quantization-aware training, and the gradients and weights of the mixed-precision quantization model are updated. This is conducive to improving the training speed of the mixed-precision quantization model.

[0089] In a possible example, the above method may also include the following steps: iteratively performing the quantization-aware training on the mixed precision quantization model; after each quantization-aware training is completed, obtaining a verification data set, and verifying the mixed precision quantization model after the quantization-aware training based on the verification data set to obtain the accuracy of the mixed precision quantization model after the quantization-aware training; if the accuracy meets the preset requirements, stopping the iterative quantization-aware training to obtain the target model after the iterative quantization-aware training.

[0090] Among them, the above preset requirements can be set by the user or by system default, and are not limited here; the preset requirements that the accuracy needs to meet can be determined based on the application of the model or the amount of data.

[0091] The verification data set may be different from the training data, and the verification data set may be used to verify the accuracy of the mixed-precision quantization model.

[0092] Among them, such as Figure 3 As shown, it is a flow chart of a quantization-aware training method, in which the training data are input into the neural network model M1 and the mixed precision quantization model M2 respectively, so as to perform a second forward inference calculation on the neural network model M1 and the mixed precision quantization model M2 respectively to obtain a first model and a second model; then, the target loss function of the mixed precision quantization model M2 or the second model is determined; according to the target loss function, the gradient and weight corresponding to the mixed precision quantization model M2 are updated; after each quantization-aware training is completed, a verification data set is obtained, and the mixed precision quantization model M2 after the quantization-aware training is verified according to the verification data set to obtain the accuracy of the mixed precision quantization model M2 after the quantization-aware training; if the accuracy meets the preset requirements, the iterative quantization-aware training is stopped to obtain the target model after the iterative quantization-aware training.

[0093] It can be seen that in this example, the process of quantization-aware training can be iterated, and after each iterative training is completed, the accuracy of the mixed-precision quantization model is determined. After the accuracy meets the preset requirements, the iterative quantization-aware training is stopped to obtain the final target model after training; in this way, the actual application needs can be met and the model adaptability is improved.

[0094] In one possible example, the method for determining the target loss function of the mixed precision quantization model may include the following steps: obtaining a first error, wherein the first error is determined by the loss function used to train the neural network model; determining a second error between the neural network model and the corresponding weight parameters of each target network layer in the mixed precision quantization model, wherein the target network layer refers to a network layer including training parameters; after performing a second forward inference calculation on the neural network model and the mixed precision quantization model respectively, determining a third error between the neural network model and the mixed precision quantization model; and determining the target loss function of the mixed precision quantization model based on the first loss, the second error, and the third error.

[0095] Among them, the above-mentioned first error (task loss) can be determined by the loss function used to train the neural network model, or it can be the error between the mixed precision quantization model M2 and the true value.

[0096] The second error (quantization error) mentioned above refers to the difference between the weight parameters of each layer with trainable parameters in the neural network model M1 and the mixed-precision quantization model M2. This second error can also be understood as the error between the quantized weight parameters, then dequantized back to floating-point parameters, and the parameters of the neural network model M1. This error represents the loss of precision during the quantization process.

[0097] The third error may refer to the sum of feature map losses corresponding to all blocks of the neural network model M1 and the mixed precision quantization model M2.

[0098] It can be seen that in this example, the target loss function can be obtained by combining the three errors to improve the training accuracy of the mixed-precision quantization model.

[0099] In one possible example, the method for determining the third error between the neural network model and the mixed precision quantization model may include the following steps: determining the error between the activation output of each block in the neural network model and the mixed precision quantization model; and weighting the errors corresponding to multiple blocks to obtain the third error.

[0100] The above-mentioned block may refer to a combination of local areas of a single layer or several layers. In this application, it may be a combination of a convolutional layer, a batch normalization layer, an activation layer, and the like.

[0101] The error between the corresponding activation outputs of each block in the above-mentioned neural network model and the mixed precision quantization model may refer to a feature map loss, which may refer to the gap or difference between the corresponding activation outputs of each block.

[0102] Among them, such as Figure 4 As shown in the figure, it is a flow chart of a method for determining the target loss function. Figure 3 After the second forward reasoning calculation is performed on M1 and M2 respectively, the third error can be calculated, as shown in FIG. Figure 4 As shown, the above B1, ..., Bn may refer to blocks of the neural network model M1, and the above B1-q, ..., Bn-q may refer to blocks of the neural network model M1. The error L3 between B1 and its corresponding B1-q may be calculated respectively. B1 , until the error L3 between Bn and the corresponding Bn-q is calculated Bn , and sum to get the third error L3 output .

[0103] Among them, such as Figure 4As shown, the first error (L1), the second error (L2) and the third error (L3) can be directly summed or weighted summed. output ) to determine the target loss function (L) of the mixed precision quantization model.

[0104] It can be seen that in this example, the method of constructing the feature map loss in units of blocks is better than comparing the errors between the feature maps output by the neural network model and the mixed precision model layer by layer; adding the third error to the loss function is beneficial to improving the inference performance of the mixed precision model.

[0105] See also Figure 5 , Figure 5 This is a flow chart of a quantization perception training method provided in an embodiment of the present application. As shown in the figure, this quantization perception training method includes the following operations.

[0106] S501. Obtain a trained neural network model.

[0107] S502: Perform a derivative process on each network layer in the neural network model to determine the derivative corresponding to each network layer.

[0108] S503: Determine disturbance information corresponding to each network layer according to the derivative corresponding to each network layer.

[0109] S504: Obtain mixed bits of training parameters corresponding to the neural network model.

[0110] S505: Generate an objective function according to the mixed bits and the disturbance information corresponding to each network layer.

[0111] S506. Obtain the power consumed by the operator corresponding to each network layer during the first forward reasoning calculation.

[0112] S507: Taking the power corresponding to each network layer as a constraint condition and constructing a linear programming problem according to the objective function.

[0113] S508: Determine the sum of the disturbance information corresponding to each network layer to obtain comprehensive disturbance information.

[0114] S509: Determine the target bit corresponding to each network layer when the comprehensive disturbance information is minimum, and determine the sum of the disturbance information corresponding to each network layer to obtain comprehensive disturbance information.

[0115] S510. Determine, according to the linear programming problem, a target bit position corresponding to each network layer when the comprehensive disturbance information is minimized.

[0116] S511. According to the neural network model, iteratively perform the quantization-aware training on the mixed-precision quantization model.

[0117] S512. After each quantization-aware training is completed, a verification data set is obtained, and the mixed-precision quantization model after the quantization-aware training is verified according to the verification data set to obtain the accuracy of the mixed-precision quantization model after the quantization-aware training.

[0118] S513: If the accuracy meets the preset requirement, stop the iterative quantization-aware training and obtain the target model after the iterative quantization-aware training.

[0119] Among them, the above steps S501-S513 can refer to Figure 2 The steps and related descriptions of the described quantization-aware training method are not repeated here.

[0120] It can be seen that the quantization-aware training method and related devices described in the embodiments of the present application obtain a trained neural network model; perform derivative processing on each network layer in the neural network model to determine the derivative corresponding to each network layer; determine the disturbance information corresponding to each network layer based on the derivative corresponding to each network layer; obtain the mixed bit positions of the training parameters corresponding to the neural network model; generate an objective function based on the mixed bit positions and the disturbance information corresponding to each network layer; obtain the power consumed by the operator corresponding to each network layer during the first forward reasoning calculation; use the power corresponding to each network layer as a constraint condition, and construct a linear programming problem based on the objective function; determine the sum of the disturbance information corresponding to each network layer to obtain comprehensive disturbance information; determine the target bit position corresponding to each network layer when the comprehensive disturbance information is minimized. The sum of the disturbance information corresponding to each network layer is determined to obtain comprehensive disturbance information; the target bit position corresponding to each network layer when the comprehensive disturbance information is minimized is determined based on the linear programming problem; the mixed-precision quantization model is iteratively trained based on the neural network model; after each quantization-aware training, a validation dataset is obtained and the mixed-precision quantization model after quantization-aware training is verified based on the validation dataset to obtain the accuracy of the mixed-precision quantization model after quantization-aware training; if the accuracy meets a preset requirement, the iterative quantization-aware training is terminated to obtain the target model after the iterative quantization-aware training. In this way, the target bit position can be obtained based on the derivative information corresponding to each network layer, which is beneficial for reducing the optimization time used to determine the mixed-precision bit position setting. In actual deployment, a linear programming problem is constructed based on constraints to determine the target bit position for each layer of the mixed-precision quantization model. In addition, the type and number of constraints are not restricted, which is beneficial for better adapting to hardware deployment requirements such as power, thereby improving the inference performance of the mixed-precision quantization model after quantization-aware training.

[0121] See also Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. As shown in the figure, the electronic device includes a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs are configured to cause the processor to execute the following steps:

[0122] Get the trained neural network model;

[0123] Constructing a linear programming problem for the neural network model, wherein the linear programming problem is used to determine the target bit corresponding to each network layer in the neural network model;

[0124] quantizing each network layer according to a target bit position corresponding to each network layer to obtain a mixed precision quantization model;

[0125] According to the neural network model, quantization-aware training is performed on the mixed precision quantization model.

[0126] It can be seen that the electronic device described in the embodiment of the present application obtains a trained neural network model; constructs a linear programming problem for the neural network model, wherein the linear programming problem is used to determine the target bit corresponding to each network layer in the neural network model; performs quantization processing on each network layer according to the target bit corresponding to each network layer to obtain a mixed precision quantization model; performs quantization perception training on the mixed precision quantization model according to the neural network model. In this way, quantization processing can be performed based on the target bit corresponding to each network layer, which is conducive to improving the optimization time used to determine the mixed precision bit setting; in actual deployment, the linear programming problem is used to constrain the actual target bit corresponding to each network layer, which is conducive to better performance adaptation with hardware deployment requirements such as power, so that after the mixed precision quantization model is quantized and trained, it is conducive to improving the inference performance of the mixed precision quantization model.

[0127] In one possible example, in constructing a linear programming problem for the neural network model, the program includes instructions for executing the following steps:

[0128] Performing a derivative process on each network layer in the neural network model to determine the derivative corresponding to each network layer;

[0129] According to the derivatives corresponding to each network layer, a linear programming problem is constructed for the neural network model.

[0130] In one possible example, in constructing a linear programming problem for the neural network model according to the derivatives corresponding to each network layer, the program includes instructions for performing the following steps:

[0131] Determining disturbance information corresponding to each network layer according to the derivative corresponding to each network layer;

[0132] Obtaining mixed bits of training parameters corresponding to the neural network model;

[0133] The linear programming problem is constructed according to the mixed bits and the disturbance information corresponding to each network layer.

[0134] In one possible example, in constructing the linear programming problem based on the mixed bits and the disturbance information corresponding to each network layer, the program includes instructions for performing the following steps:

[0135] generating an objective function according to the mixed bit and the disturbance information corresponding to each network layer;

[0136] Obtaining the power consumed by the operator corresponding to each network layer during the first forward reasoning calculation;

[0137] The power corresponding to each network layer is used as a constraint condition, and a linear programming problem is constructed according to the objective function.

[0138] In one possible example, in determining the target bit corresponding to each network layer in the neural network model, the program includes instructions for performing the following steps:

[0139] Determining the sum of disturbance information corresponding to each network layer to obtain comprehensive disturbance information;

[0140] The target bit position corresponding to each network layer when the comprehensive disturbance information is minimized is determined according to the linear programming problem.

[0141] In one possible example, in performing quantization-aware training on the mixed-precision quantization model according to the neural network model, the program includes instructions for performing the following steps:

[0142] Inputting the training data into the neural network model and the mixed precision quantization model respectively, so as to perform a second forward reasoning calculation on the neural network model and the mixed precision quantization model respectively, to obtain a first model and a second model;

[0143] Determining a target loss function for the mixed precision quantization model;

[0144] According to the target loss function, the gradient and weight corresponding to the mixed precision quantization model are updated.

[0145] In one possible example, the program further includes instructions for executing the following steps:

[0146] Iteratively performing the quantization-aware training on the mixed-precision quantization model;

[0147] After each quantization-aware training is completed, obtaining a verification data set, and verifying the mixed-precision quantization model after the quantization-aware training according to the verification data set to obtain the accuracy of the mixed-precision quantization model after the quantization-aware training;

[0148] If the accuracy meets the preset requirement, the iterative quantization-aware training is stopped to obtain the target model after the iterative quantization-aware training.

[0149] In one possible example, in determining the target loss function of the mixed precision quantization model, the program includes instructions for performing the following steps:

[0150] Obtaining a first error, wherein the first error is determined by a loss function used to train the neural network model;

[0151] Determining a second error between the neural network model and a corresponding weight parameter of each target network layer in the mixed precision quantization model, wherein the target network layer refers to a network layer including training parameters;

[0152] After performing the second forward reasoning calculation on the neural network model and the mixed precision quantization model respectively, determining a third error between the neural network model and the mixed precision quantization model;

[0153] Determine a target loss function of the mixed precision quantization model according to the first loss, the second error, and the third error.

[0154] In one possible example, in determining the third error between the neural network model and the mixed precision quantization model, the program includes instructions for performing the following steps:

[0155] Determining the error between the activation outputs of each block in the neural network model and the mixed precision quantization model;

[0156] The errors corresponding to the multiple blocks are weighted to obtain the third error.

[0157] The above mainly introduces the solution of the embodiment of the present application from the perspective of the execution process of the method side. It is understandable that, in order to realize the above functions, the electronic device includes a hardware structure and / or software module corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiment provided herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software driven hardware manner depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0158] The embodiment of the present application can divide the functional units of the electronic device according to the above method example. For example, each functional unit can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0159] In the case of dividing each functional module into corresponding functional modules, Figure 7A A schematic diagram of a quantization-aware training device is shown in FIG. Figure 7A As shown, the quantization perception training device 700 may include: an acquisition unit 701, a construction unit 702, a quantization processing unit 703 and a quantization perception training unit 704, wherein:

[0160] The acquisition unit 701 is used to acquire a trained neural network model;

[0161] The construction unit 702 is configured to construct a linear programming problem for the neural network model, wherein the linear programming problem is used to determine a target bit corresponding to each network layer in the neural network model;

[0162] The quantization processing unit 703 is configured to perform quantization processing on each network layer according to the target bit corresponding to each network layer to obtain a mixed precision quantization model;

[0163] The quantization-aware training unit 704 is configured to perform quantization-aware training on the mixed-precision quantization model based on the neural network model.

[0164] It can be seen that the quantization-aware training device described in the embodiment of the present application obtains a trained neural network model; constructs a linear programming problem for the neural network model, wherein the linear programming problem is used to determine the target bit corresponding to each network layer in the neural network model; performs quantization processing on each network layer according to the target bit corresponding to each network layer to obtain a mixed precision quantization model; and performs quantization-aware training on the mixed precision quantization model according to the neural network model. In this way, quantization processing can be performed based on the target bit corresponding to each network layer, which is conducive to improving the optimization time used to determine the mixed precision bit setting; in actual deployment, the linear programming problem is used to constrain the actual target bit corresponding to each network layer, which is conducive to better performance adaptation with hardware deployment requirements such as power, so that after the mixed precision quantization model is quantized and trained, it is conducive to improving the inference performance of the mixed precision quantization model.

[0165] In a possible example, in constructing a linear programming problem for the neural network model, the constructing unit 702 is specifically configured to:

[0166] Performing a derivative process on each network layer in the neural network model to determine the derivative corresponding to each network layer;

[0167] According to the derivatives corresponding to each network layer, a linear programming problem is constructed for the neural network model.

[0168] In a possible example, in constructing a linear programming problem for the neural network model according to the derivatives corresponding to each network layer, the constructing unit 702 is specifically configured to:

[0169] Determining disturbance information corresponding to each network layer according to the derivative corresponding to each network layer;

[0170] Obtaining mixed bits of training parameters corresponding to the neural network model;

[0171] The linear programming problem is constructed according to the mixed bits and the disturbance information corresponding to each network layer.

[0172] In a possible example, in constructing the linear programming problem according to the mixed bits and the disturbance information corresponding to each network layer, the constructing unit 702 is specifically configured to:

[0173] generating an objective function according to the mixed bit and the disturbance information corresponding to each network layer;

[0174] Obtaining the power consumed by the operator corresponding to each network layer during the first forward reasoning calculation;

[0175] The power corresponding to each network layer is used as a constraint condition, and a linear programming problem is constructed according to the objective function.

[0176] In one possible example, Figure 7B A schematic diagram of another quantitative perception training device is shown. Figure 7A In comparison, the quantization-aware training device 700 may further include: a determination unit 705, wherein, in determining the target bit corresponding to each network layer in the neural network model, the determination unit 705 is used to: determine the sum of the disturbance information corresponding to each network layer to obtain comprehensive disturbance information; and determine, according to the linear programming problem, the target bit corresponding to each network layer when the comprehensive disturbance information is minimized.

[0177] In a possible example, in performing quantization-aware training on the mixed-precision quantization model according to the neural network model, the quantization-aware training unit 704 is specifically configured to:

[0178] Inputting the training data into the neural network model and the mixed precision quantization model respectively, so as to perform a second forward reasoning calculation on the neural network model and the mixed precision quantization model respectively, to obtain a first model and a second model;

[0179] Determining a target loss function for the mixed precision quantization model;

[0180] According to the target loss function, the gradient and weight corresponding to the mixed precision quantization model are updated.

[0181] In a possible example, the quantization-aware training unit 704 is specifically configured to:

[0182] Iteratively performing the quantization-aware training on the mixed-precision quantization model;

[0183] After each quantization-aware training is completed, obtaining a verification data set, and verifying the mixed-precision quantization model after the quantization-aware training according to the verification data set to obtain the accuracy of the mixed-precision quantization model after the quantization-aware training;

[0184] If the accuracy meets the preset requirement, the iterative quantization-aware training is stopped to obtain the target model after the iterative quantization-aware training.

[0185] In a possible example, in determining the target loss function of the mixed precision quantization model, the determining unit 705 is further configured to:

[0186] Obtaining a first error, wherein the first error is determined by a loss function used to train the neural network model;

[0187] Determining a second error between the neural network model and a corresponding weight parameter of each target network layer in the mixed precision quantization model, wherein the target network layer refers to a network layer including training parameters;

[0188] After performing the second forward reasoning calculation on the neural network model and the mixed precision quantization model respectively, determining a third error between the neural network model and the mixed precision quantization model;

[0189] Determine a target loss function of the mixed precision quantization model according to the first loss, the second error, and the third error.

[0190] In a possible example, in determining the third error between the neural network model and the mixed precision quantization model, the determining unit 705 is further configured to:

[0191] Determining the error between the activation outputs of each block in the neural network model and the mixed precision quantization model;

[0192] The errors corresponding to the multiple blocks are weighted to obtain the third error.

[0193] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0194] The electronic device provided in this embodiment is used to execute the above-mentioned device communication method, and thus can achieve the same effect as the above-mentioned implementation method.

[0195] When integrated units are used, the electronic device may include a processing module, a storage module, and a communication module. The processing module may be used to control and manage the actions of the electronic device. For example, it may be used to support the electronic device in executing the steps performed by the acquisition unit 701, construction unit 702, quantization processing unit 703, quantization-aware training unit 704, and determination unit 705. The storage module may be used to support the electronic device in executing and storing program code and data. The communication module may be used to support communication between the electronic device and other electronic devices.

[0196] The processing module may be a processor or a controller. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, and so on. The storage module may be a memory. The communication module may specifically be a device that interacts with other electronic devices, such as a radio frequency circuit, a Bluetooth chip, or a Wi-Fi chip.

[0197] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute part or all of the steps of any method described in the above method embodiments, and the above computer includes an electronic device.

[0198] The present application also provides a computer program product comprising a non-transitory computer-readable storage medium storing a computer program, wherein the computer program is operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments. The computer program product may be a software installation package, and the computer may comprise an electronic device.

[0199] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0200] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0201] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0202] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0203] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0204] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes a number of instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the above-mentioned methods of each embodiment of the present application. The aforementioned memory includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0205] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, which may include: a flash drive, a read-only memory, a random access memory, a magnetic disk or an optical disk, etc.

[0206] The above is a detailed introduction to the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for those skilled in the art, according to the idea of ​​the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A quantitative perception training method, characterized in that: include: Obtaining a trained neural network model, where the neural network model is trained using training data, where the training data is image data or speech data; Performing a derivative process on each network layer in the neural network model to determine the derivative corresponding to each network layer; Constructing a linear programming problem for the neural network model based on the derivatives corresponding to each network layer, wherein the linear programming problem is used to determine the target bit corresponding to each network layer in the neural network model; quantizing each network layer according to a target bit position corresponding to each network layer to obtain a mixed precision quantization model; According to the neural network model, quantization-aware training is performed on the mixed precision quantization model.

2. The method according to claim 1, characterized in that Constructing a linear programming problem for the neural network model according to the derivatives corresponding to each network layer includes: Determining disturbance information corresponding to each network layer according to the derivative corresponding to each network layer; Obtaining mixed bits of training parameters corresponding to the neural network model; The linear programming problem is constructed according to the mixed bits and the disturbance information corresponding to each network layer.

3. The method according to claim 2, characterized in that The constructing the linear programming problem according to the mixed bit and the disturbance information corresponding to each network layer includes: generating an objective function according to the mixed bit and the disturbance information corresponding to each network layer; Obtaining the power consumed by the operator corresponding to each network layer during the first forward reasoning calculation; The power corresponding to each network layer is used as a constraint condition, and a linear programming problem is constructed according to the objective function.

4. The method according to claim 3, characterized in that Determining the target bit corresponding to each network layer in the neural network model includes: Determining the sum of disturbance information corresponding to each network layer to obtain comprehensive disturbance information; The target bit position corresponding to each network layer when the comprehensive disturbance information is minimized is determined according to the linear programming problem.

5. The method according to any one of claims 1 to 4, characterized in that The performing quantization-aware training on the mixed-precision quantization model according to the neural network model includes: Inputting the training data into the neural network model and the mixed precision quantization model respectively, so as to perform a second forward reasoning calculation on the neural network model and the mixed precision quantization model respectively; Determining a target loss function for the mixed precision quantization model; According to the target loss function, the gradient and weight corresponding to the mixed precision quantization model are updated.

6. The method according to claim 5, characterized in that The method further comprises: Performing iterative quantization-aware training on the mixed-precision quantization model; After each quantization-aware training is completed, obtaining a verification data set, and verifying the mixed-precision quantization model after the quantization-aware training according to the verification data set to obtain the accuracy of the mixed-precision quantization model after the quantization-aware training; If the accuracy meets the preset requirement, the iterative quantization-aware training is stopped to obtain the target model after the iterative quantization-aware training.

7. The method according to claim 5, characterized in that Determining the target loss function of the mixed precision quantization model includes: Obtaining a first error, wherein the first error is determined by a loss function used to train the neural network model; Determining a second error between the neural network model and a corresponding weight parameter of each target network layer in the mixed precision quantization model, wherein the target network layer refers to a network layer including training parameters; After performing the second forward reasoning calculation on the neural network model and the mixed precision quantization model respectively, determining a third error between the neural network model and the mixed precision quantization model; Determine a target loss function of the mixed precision quantization model according to the first error, the second error, and the third error.

8. The method according to claim 7, characterized in that Determining a third error between the neural network model and the mixed precision quantization model includes: Determining the error between the activation outputs of each block in the neural network model and the mixed precision quantization model; The errors corresponding to the multiple blocks are weighted to obtain the third error.

9. A quantitative perception training device, characterized in that: The device includes: an acquisition unit, a construction unit, a quantization processing unit and a quantization perception training unit, wherein: The acquisition unit is used to acquire a trained neural network model, where the neural network model is trained with training data, where the training data is image data or speech data; The construction unit is configured to perform a derivative process on each network layer in the neural network model to determine a derivative corresponding to each network layer; and construct a linear programming problem for the neural network model based on the derivative corresponding to each network layer, wherein the linear programming problem is used to determine a target bit corresponding to each network layer in the neural network model; The quantization processing unit is used to perform quantization processing on each network layer according to the target bit corresponding to each network layer to obtain a mixed precision quantization model; The quantization-aware training unit is used to perform quantization-aware training on the mixed-precision quantization model based on the neural network model.

10. An electronic device, characterized in that: The method comprises a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for executing the steps in the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that A computer program for electronic data exchange is stored, wherein the computer program enables a computer to execute the method according to any one of claims 1 to 8.

12. A computer program product, wherein: The computer program product comprises a non-transitory computer-readable storage medium storing a computer program, wherein the computer program is operable to cause a computer to execute the method according to any one of claims 1 to 8.

Images