Image recognition method and system based on adaptive data quantization and polyhedral template

Abstract

The present invention relates to an image recognition method and system based on adaptive data quantization and polyhedron templates. The method comprises: according to a neural network encoded in a traditional data format, using a search algorithm based on Bayesian optimization to determine the optimal value set of AFP; according to the optimal value set, encoding the neural network in AFP format to generate a neural network after dynamic quantization encoding; using a code template with optimal parameters to describe the data calculation method of each layer of the neural network after dynamic quantization encoding, and generating an executable code of the neural network after dynamic quantization encoding; obtaining image data encoded in AFP format; deploying the executable code to the end-side device to identify the image data and obtain an image recognition result. The method of the present invention accelerates the computational efficiency of the neural network in the image recognition process through quantization technology and a polyhedron-based search strategy, improves the computational accuracy, and saves equipment resources.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to an image recognition method and system based on adaptive data quantization and polyhedron templates. Background Art

[0002] In recent years, with the explosive growth in the number and types of edge devices, the combination of deep neural networks (DNN) and various edge devices has brought about numerous applications, such as Tesla's autonomous driving, Alibaba Cloud's City Brain, and face recognition. However, deep neural networks have many parameters and many layers, which place huge demands on the computing resources of edge devices (such as device energy consumption and memory). Therefore, due to the limited resources of edge devices, how to accelerate the operation of neural networks on edge devices has become a very important issue.

[0003] In order to save computing resources, DNN compression technology has been widely used. An important branch of DNN compression technology is quantization. The so-called quantization refers to converting the continuously distributed values ​​in the neural network into discretely distributed values, such as representing float32 data as int8 data, so as to use a lower bit width encoding method to represent the data. The lower bit width encoding method can reduce the space for data storage and the amount of data carried on the bus, and can also effectively reduce the overhead of operand calculation, thereby reducing energy consumption. Low bit width operands can reduce the load of the main operators (i.e.: multiplication and accumulation operations) in neural network calculations, so they can accelerate the calculation process of the neural network. The commonly used method is to use a fixed length data format (sign bit, exponent bit, mantissa bit) to perform data quantization operations. Given a batch of data, the distribution of the data may be concentrated in a certain interval, which leads to the difficulty of fixed bit width quantization in the data representation range, insufficient representation accuracy, and inability to adapt to changes in data adaptation, thereby affecting the accuracy and efficiency of neural network calculations.

[0004] In order to speed up the calculation on the specified hardware, it is also important to map the quantized data on the hardware architecture. For example, on the x86 CPU architecture, in order to make full use of the locality of access data between memory and cache, the size of the data block is generally designed to be the size of the cacheline. In general, programmers manually optimize for different underlying hardware architectures, but this optimization method requires a lot of manpower. Therefore, template-based search strategies have been widely used. The so-called template-based search strategy means that the programmer gives a certain search space, which contains a combination of various parameters (block size, loop order, etc.), and then searches based on a certain search strategy (profile, cost model, etc.) to find the optimal solution. However, in the search process, the template-based search strategy will produce an explosive search space, and there are still certain limitations in the description of the search template. Therefore, this leads to the optimal solution in the search process and the actual search cost may not match. In other words, spending a lot of search cost to search in a huge search space is not only time-consuming and resource-consuming, but also may not necessarily search for the optimal solution.

[0005] Different application scenarios of neural networks are determined by their structure (number of layers, etc.). Therefore, different neural networks can be applied in different scenarios, such as image recognition, object detection, etc. Since the traditional quantitative data format has the problem of waste of accuracy and representation range on the data set, and the traditional template-based search strategy has the limitation of description method, the mismatch between search effect and search strategy, when neural networks are applied to image recognition of faces or objects, there are problems of low calculation accuracy and efficiency, and serious waste of resources. Summary of the invention

[0006] The purpose of the present invention is to provide an image recognition method and system based on adaptive data quantization and polyhedral templates, so as to jointly accelerate the computing efficiency of neural networks, improve computing accuracy and save equipment resources when performing image recognition.

[0007] To achieve the above object, the present invention provides the following solutions:

[0008] An image recognition method based on adaptive data quantization and polyhedral templates, comprising:

[0009] Obtaining a neural network encoded in a traditional data format selected by the end-side device; the traditional data format includes an FP32 data format;

[0010] According to the neural network encoded in the traditional data format, an optimal value set of a floating point representation AFP of the adaptive data is determined by using a search algorithm based on Bayesian optimization; the optimal value set includes an exponent bit width, a mantissa bit width and an offset value corresponding to each layer in the neural network;

[0011] According to the optimal value set, encoding the neural network in the AFP format to generate a dynamically quantized encoded neural network;

[0012] Acquire a code template based on polyhedron technology and a memory size of the terminal device;

[0013] Determining optimal parameters of the code template according to the memory size of the terminal device; the optimal parameters include data block size;

[0014] Using a code template with the optimal parameters to describe the data calculation method of each layer of the neural network after the dynamic quantization encoding, and generating an executable code of the neural network after the dynamic quantization encoding;

[0015] Acquire image data encoded in the AFP format; the image data includes face image data and object image data;

[0016] The executable code is deployed on the terminal device to recognize the image data and obtain an image recognition result.

[0017] Optionally, the determining the optimal value set of AFP according to the neural network encoded in the traditional data format by using a search algorithm based on Bayesian optimization specifically includes:

[0018] Generate a candidate set of multiple possible values ​​according to a Bayesian optimization algorithm;

[0019] The optimal value set is selected from the candidate set according to a preset calculation function.

[0020] Optionally, determining the optimal parameters of the code template according to the memory size of the terminal device specifically includes:

[0021] Determine a candidate set of the search space according to the memory size of the terminal device;

[0022] Using an integer set library analyzed based on polyhedron technology, taking the code template as input, and outputting a scheduling tree representing a computational relationship;

[0023] Based on different scheduling trees, a two-step search strategy is adopted to determine the optimal parameters in the candidate set of the search space.

[0024] Optionally, the acquiring the image data encoded in the AFP format specifically includes:

[0025] Collecting an image received by the terminal device; the image includes a face image or an object image;

[0026] The image is encoded in the AFP format to generate image data encoded in the AFP format.

[0027] An image recognition system based on adaptive data quantization and polyhedral templates, comprising:

[0028] A traditional encoding neural network acquisition module, used to acquire a neural network encoded in a traditional data format selected by a terminal device; the traditional data format includes an FP32 data format;

[0029] A Bayesian optimization search module is used to determine an optimal value set of a floating point expression form AFP of adaptive data according to the neural network encoded in the traditional data format by using a search algorithm based on Bayesian optimization; the optimal value set includes an exponent bit width, a mantissa bit width and an offset value corresponding to each layer in the neural network;

[0030] An AFP quantization encoding module is used to encode the neural network in the AFP format according to the optimal value set to generate a neural network after dynamic quantization encoding;

[0031] A code template acquisition module, used to acquire a code template based on polyhedron technology and a memory size of the terminal device;

[0032] An optimal parameter determination module, used to determine the optimal parameters of the code template according to the memory size of the terminal device; the optimal parameters include the data block size;

[0033] An executable code generation module, used to describe the data calculation method of each layer of the neural network after the dynamic quantization encoding by using a code template with the optimal parameters, and generate an executable code of the neural network after the dynamic quantization encoding;

[0034] An image data acquisition module, used to acquire image data encoded in the AFP format; the image data includes face image data and object image data;

[0035] The image recognition module is used to deploy the executable code to the terminal device to recognize the image data and obtain an image recognition result.

[0036] Optionally, the Bayesian optimization search module specifically includes:

[0037] A candidate set generation unit, used to generate a plurality of candidate sets of possible values ​​according to a Bayesian optimization algorithm;

[0038] The optimal value set selection unit is used to select the optimal value set from the candidate set according to a preset calculation function.

[0039] Optionally, the optimal parameter determination module specifically includes:

[0040] A search space determination unit, configured to determine a candidate set of the search space according to the memory size of the terminal device;

[0041] A scheduling tree output unit, used to use the integer set library analyzed based on polyhedron technology, take the code template as input, and output a scheduling tree representing the calculation relationship;

[0042] The optimal parameter determination unit is used to determine the optimal parameter in the candidate set of the search space by adopting a two-step search strategy based on different scheduling trees.

[0043] Optionally, the image data acquisition module specifically includes:

[0044] An image acquisition unit, configured to acquire an image received by the terminal device; the image includes a face image or an object image;

[0045] The image AFP encoding unit is used to encode the image in the AFP format to generate image data encoded in the AFP format.

[0046] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0047] The present invention provides an image recognition method and system based on adaptive data quantization and polyhedron templates, the method comprising: obtaining a neural network encoded in a traditional data format selected by a terminal device; the traditional data format comprises an FP32 data format; according to the neural network encoded in the traditional data format, using a search algorithm based on Bayesian optimization to determine an optimal value set of a floating point expression form AFP of adaptive data; the optimal value set comprises an exponent bit width, a mantissa bit width and an offset value corresponding to each layer in the neural network; according to the optimal value set, encoding the neural network in the AFP format to generate a neural network after dynamic quantization encoding; obtaining a code template based on polyhedron technology and a memory size of the terminal device; determining the optimal parameters of the code template according to the memory size of the terminal device; the optimal parameters comprise a data block size; using a code template with the optimal parameters to describe a data calculation method for each layer of the neural network after dynamic quantization encoding, generating an executable code of the neural network after dynamic quantization encoding; obtaining image data encoded in the AFP format; the image data comprises face image data and object image data; deploying the executable code on the terminal device to recognize the image data to obtain an image recognition result.

[0048] In order to solve the waste of precision and representation range of traditional quantized data formats on data sets, the present invention proposes a solution of quantizing using the floating-point expression AFP of adaptive data; at the same time, in order to solve the limitations of the description method, the mismatch between search effect and search strategy of traditional TVM-based templates, the present invention proposes code template design and search strategy design based on polyhedron technology; finally, the present invention accelerates the computational efficiency of neural networks in the image recognition process through quantization technology and polyhedron-based search strategy, improves computational accuracy, and saves equipment resources (energy consumption, etc.). BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0050] Figure 1 A flowchart of an image recognition method based on adaptive data quantization and polyhedron template provided by an embodiment of the present invention;

[0051] Figure 2 A schematic diagram of the principle of a neural network acceleration solution based on adaptive data quantization and polyhedral templates provided in an embodiment of the present invention;

[0052] Figure 3 A schematic diagram of a traditional quantized data format provided by an embodiment of the present invention;

[0053] Figure 4 A schematic diagram of an adaptive data quantization algorithm provided by an embodiment of the present invention;

[0054] Figure 5 A schematic diagram of an algorithm of a two-layer for loop provided in an embodiment of the present invention;

[0055] Figure 6 A schematic diagram of an algorithm after affine transformation provided by an embodiment of the present invention;

[0056] Figure 7 A schematic diagram of a code template provided by an embodiment of the present invention;

[0057] Figure 8 A structural diagram of an image recognition system based on adaptive data quantization and polyhedron template provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] The purpose of the present invention is to provide an image recognition method and system based on adaptive data quantization and polyhedral templates, so as to jointly accelerate the computing efficiency of neural networks, improve computing accuracy and save equipment resources when performing image recognition.

[0060] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0061] Figure 1 A flowchart of an image recognition method based on adaptive data quantization and polyhedron template is provided in an embodiment of the present invention. Figure 2 A schematic diagram of the principle of a neural network acceleration solution based on adaptive data quantization and polyhedral templates provided in an embodiment of the present invention. Figure 1 and Figure 2 The present invention provides an image recognition method based on adaptive data quantization and polyhedron template, comprising:

[0062] Step 101: Obtain a neural network encoded in a traditional data format selected by a terminal device.

[0063] The calculation of neural network includes reasoning and training. In the calculation of neural network, a large number of multiplication and addition operations are involved. In order to reduce the multiplication and addition operations, there are five commonly used traditional data formats. Figure 3 The following is a schematic diagram of a traditional quantized data format provided by an embodiment of the present invention. Figure 3 As shown, traditional data formats include INT8, FP32, FP16, BFP16, and TF32.

[0064] FP32, TF32, FP16 and BF16 are all floating-point formats. The floating-point format divides the data string into three parts: the sign bit, the exponent bit and the mantissa bit. The sign bit is used to indicate whether the current data is positive or negative; the exponent bit is used to indicate the range of the data; and the mantissa bit is used to indicate the accuracy of the data. For example, given an FP32 data b fp32 = {b 31 , b 30 , …, b23 , b 22 ,…,b0}; b i ∈{0, 1}. Sign bit b 31 Occupies 1 bit, exponent bit b 30 -b 23 Occupies 8 bits, the last digit is b 22 -b0 occupies 23 bits. The data range it expresses is calculated according to the following formula:

[0065]

[0066] Therefore, the data range of the expression mainly depends on the exponent bits. The longer the exponent bits, the larger the data range. The accuracy mainly depends on the mantissa bits. The longer the mantissa bits, the greater the accuracy. At the same time, the number of multiplication and addition calculations is closely related to the bit width. The shorter the bit width, the fewer the number of multiplication and addition calculations.

[0067] In the current existing data representation method, the bit width of the exponent and mantissa bits is fixed, which means that the current data format has a fixed representation range. However, for different neural networks, the distribution of parameters in each layer is different. According to observations, the current data distribution approximately fits the Gaussian distribution, that is, the distribution of data is concentrated in a certain range. Inside this range, the data distribution is relatively dense, and outside this range, the data distribution is relatively sparse. Therefore, this requires the use of higher precision (longer mantissa bits) within the densely distributed range. Therefore, the current fixed representation method cannot adapt to the distribution of data, which leads to low efficiency in matching data representation and data distribution, thereby affecting the low accuracy of the final neural network calculation.

[0068] At the same time, the existing fixed-bit width representation method still has round-off errors. Round-off error refers to the difference between the approximate value and the exact value obtained by the operation. For example, BF16 is used to represent FP32 data. In terms of data representation range, because the exponent bits of the two are the same, the integer range represented by FP32 and BF16 is the same. However, the mantissa bits of FP32 are longer than the mantissa bits of BF16, and the precision of FP32 is greater than that of BF16. The precision error represented by its decimal part leads to round-off errors. In addition, when FP16 is used to represent FP32 data, there is a risk of overflow. The data range represented by FP32 and FP16 is different. In big data computing, FP16 has an overflow risk. The dynamic range of FP16 (6*10 -8 ~65504) is much lower than the dynamic range of FP32 (1.4*10 -45 ~1.7*10 38 ), precision(2 -10 ) is much lower than the accuracy of FP32 (2 -23).

[0069] Step 102: According to the neural network encoded in the traditional data format, a search algorithm based on Bayesian optimization is used to determine the optimal value set of the floating point expression form AFP of the adaptive data.

[0070] Model quantization refers to converting the continuously distributed values ​​in the neural network into discretely distributed values. According to the quantization method, quantization is divided into two types: uniform quantization and non-uniform quantization.

[0071] Uniform quantization refers to uniformly mapping continuously distributed data to discrete space. In general, the uniform quantization method defines the range of values ​​in discrete space [min, max], and then maps the continuously distributed values ​​to the nearest integer. For example, all floating point numbers can be sparsely scaled to the range of [-128, 127] of INT8 according to scaling.

[0072] Non-uniform quantization refers to mapping continuously distributed data to a variable range based on the characteristics of the data. For example, clustering algorithms analyze the data values, select the data values ​​that best represent the current distribution, and then quantize based on the current data values. For example, by compressing the exponent bits and mantissa bits in FP32.

[0073] Among them, uniform distribution is not suitable for the current neural network. Because in the convolution layer of the neural network, the numerical distribution of the parameters is approximately fitted to the Gaussian distribution (W~N(μ,δ 2 ), where μ is the mean, δ 2 is the variance. That is to say, most of the values ​​are basically distributed around 0, and there are basically no parameters with very large absolute values. Therefore, using uniform distribution again will make most of the values ​​concentrated around an integer, which will lead to the accumulation of calculation errors and ultimately cause the correctness of the neural network to drop sharply. At the same time, the quantization of uniform distribution generally converts a certain type into a low-bit data type. During the transmission process of the network layer, uniform quantization needs to be requantized for the continuity of calculation. For example, for convolution, the input is FP32 and the weight is FP32. First, quantize the input and weight to INT8, use INT8 data for multiplication and addition, and finally dequantize the result of INT8 to FP32 and pass it to the subsequent BatchNorm layer for calculation.

[0074] The data type of non-uniform quantization is still floating point (BF16, etc.), which can reduce the error caused by quantization. However, in the existing floating point data representation, the bit width of the exponent bit and the mantissa bit is fixed. This fixed representation cannot adapt to the distribution of data, which leads to low efficiency in matching data representation and data distribution, thus affecting the low accuracy of the final neural network calculation. At the same time, the existing fixed bit width representation also has round-off errors.

[0075] In order to solve the waste of precision and representation range of traditional quantized data formats on data sets, the present invention proposes a solution that can perform quantization in an adaptive data format.

[0076] In this step 102, the input is a neural network encoded in a traditional data format, such as a neural network with a depth of L layers encoded in the traditional FP32 data format. For each layer of data in the network represented by FP32, the bit width of its exponent bit and mantissa bit is usually determined. However, when the distribution of data is densely concentrated in a certain interval, too many exponent bits are not needed. Instead, the mantissa bits should be increased to increase the accuracy, thereby reducing the precision error and rounding error introduced in the quantization process.

[0077] Therefore, in order to improve the above problems, the present invention proposes a set of adaptive floating-point representations of data (Adaptive Floating-Point, AFP), which can dynamically adjust the exponent bits and mantissa bits according to the distribution of data.

[0078] For each layer of data in the network, the floating point representation of the adaptive data is b AFP By the sign bit (1 bit), exponent bit (n exp bit), mantissa bit (n man bit). exp 、n man The bit width is adjusted according to the actual data of each layer. The specific data encoding format is as follows:

[0079]

[0080] For a floating point number encoded as a binary string of 0s and 1s, the value of the floating point data represented by this data string can be obtained according to the following calculation formula:

[0081]

[0082] Therefore, compared with the traditional FP32 fixed bit width encoding format, the present invention can dynamically adjust the number of exponent bits, mantissa bits and the size of the offset value k according to the distribution of data, thereby reducing the error in the quantization process. Therefore, after the AFP encoding of the neural network, the parameters of each layer are actually a coding bit width (n exp , n man ) and the set of offset values ​​k (n exp , n man , k).

[0083] For the set of encoding bit width and offset values ​​(n exp , n man , k), because the numerical distributions of different layers of the neural network are different, so the set (n exp , n man The actual value of k should be adjusted according to different layers. The following describes how to adaptively adjust the actual size of the bit width of each layer (n k) based on the designed Bayesian optimization search algorithm according to the distribution of each layer of data. exp , n man ) and the size of the offset value k.

[0084] Figure 4 Schematic diagram of an adaptive data quantization algorithm provided by an embodiment of the present invention. Figure 4 , the input of the adaptive data quantization algorithm has three:

[0085] (1) Indicates a neural network with a depth of L layers encoded in the traditional FP format. Before quantization in the AFP format, the set of floating-point numbers represented by FP32 in each layer of the network is W l .

[0086] (2) The amount of sampled data n. The present invention generates a candidate set D of n possible values ​​according to the Bayesian Optimization algorithm. n Each candidate set represents a possible values. The algorithm selects the best solution from the candidate set.

[0087] (3) The number of iterations of the algorithm N, N represents the maximum number of search iterations of the algorithm.

[0088] After the above search algorithm, there are two overall outputs:

[0089] (1) Each layer parameter (n exp , n max , the optimal value set θ of k) * .

[0090] (2) After determining the bit width and offset values ​​of each layer, the overall neural network parameters using the AFP encoding method are

[0091] The present invention needs to determine the encoding method of each layer according to the distribution of data, that is, to determine the size of the three parameters in the above encoding method (n exp , n man , k). The step 102 determines the optimal value set of AFP according to the neural network encoded in the traditional data format by using a search algorithm based on Bayesian optimization, specifically including:

[0092] Step 2.1: Generate multiple candidate sets of possible values ​​according to the Bayesian optimization algorithm;

[0093] Generate candidate set D based on Gaussian model n (See Figure 4 Line 1 of the algorithm selects the optimal parameter (n exp , n man , k)( Figure 4 Line 5 of the algorithm).

[0094] Step 2.2: Select the optimal value set from the candidate set according to a preset calculation function. The optimal value set includes the exponent bit width, mantissa bit width and offset value corresponding to each layer in the neural network.

[0095] According to the optimal parameters, the data is quantized (Line 7-Line 8), the error of the quantized data is calculated (Line 9-Line 11) and the parameters required in the entire search strategy are updated (Line 12-Line 13). At the same time, the optimal error is updated (Line 14-Line 16). When the entire algorithm is completed, the output value corresponds to the value corresponding to the optimal error.

[0096] See also Figure 4 The specific search process is as follows, where Line X represents the Xth line of the algorithm.

[0097] Line 1: This algorithm generates a candidate set D of n possible values ​​based on the Bayesian Optimization algorithm n , each candidate set represents a Possible values. This step represents the Gaussian modeling fitting target parameters based on Line 13 possible distribution of .

[0098] Line 2: Initialize the number of iterations N.

[0099] Line 3: Initialize the optimal error J. One of the quantitative evaluation indicators is the error J caused by less quantization. The present invention uses the product of Kullback-Leibler divergence (KL divergence) and the cost of the candidate set to measure the error J. Therefore, the goal of the joint search is to minimize the error J, that is:

[0100]

[0101] in The KL divergence between the quantization method of the present invention and the quantization method of FP32 is defined as follows:

[0102]

[0103] represents the cost of the current candidate set; λ l is an exponential coefficient representing the total number of addition and multiplication operations, where n is the number of multiplications required. exp +(1+n man ) 2 bits, addition requires access to The definition is as follows:

[0104]

[0105] Line 4: Sets the loop condition for the entire search algorithm.

[0106] Line 5: α represents a specific computation function. The first input θ is the parameters of the entire network, and the second input is the candidate set D n This algorithm calculates the optimal solution of the function corresponding to Assign to θ * ={k′, e′, p′}. k′ represents the size of the offset value, and e′ represents the exponent bit width n exp The size of p′ represents the width of the mantissa bit n man size.

[0107] Line 6: Calculate the total number of coded bits Nq′=1+e′+p′.

[0108] Line 7 and Line 8: Based on the number of bits (bit width) and offset value determined during this iteration, the quantized data This algorithm obtains quantized data

[0109] Line 9: According to the determined process in this iteration and Calculate the KL divergence D between the AFP quantization method of the present invention and the FP32 quantization methodKL .

[0110] Line 10: Calculate the cost C of the current candidate set based on the bit width Nq′ and e′ obtained this time.

[0111] Line 11: Calculate the new error J new = KL divergence D KL * The cost Cλ of this iteration l .

[0112] Line 12: Based on the new error J new and the original candidate set D n Generate a new candidate set D n+1 .

[0113] Line 13: Update the Gaussian model gP new .

[0114] Line 14 and Line 15: If the current latest error J new Is the optimal error (minimum), then update J. The output of the algorithm is the value corresponding to the optimal error J, including the exponent bit width, mantissa bit width and offset value corresponding to each layer in the neural network.

[0115] Step 103: According to the optimal value set, the neural network is encoded in the AFP format to generate a dynamically quantized encoded neural network.

[0116] The input processed by the present invention is a neural network with a depth of L layers encoded in the traditional FP32 data format. For the neural network encoded in FP32, the present invention uses the floating point representation of adaptive data (Adaptive Floating-Point, AFP) to encode the parameters of each layer in the neural network. The bit width set of the parameter and the set of offset values ​​are (n exp , n man , k).

[0117] For each layer parameter bit width set after encoding (n exp , n man , k), in order to determine its actual bit width length and offset value size, the present invention uses a search algorithm based on Bayesian optimization to search for bit width and offset value size. After determining the quantization data of each layer (the exponent bit width, mantissa bit width and offset value corresponding to each layer in the neural network), the present invention can finally obtain a neural network after dynamic quantization encoding with adaptive data.

[0118] After encoding the neural network in the AFP format according to the optimal value set and generating a dynamically quantized encoded neural network, in order to deploy the neural network on different hardware, different frameworks generate calculation codes for data, and the calculation rules of the code (including the order of loops, loop parameters, etc.) will affect the calculation execution efficiency of the neural network. How to generate a set of executable calculation codes that can fully utilize the hardware is very important.

[0119] TVM (Tensor Virtual Machine) is the most widely used deep neural network compiler. Its main search strategy is template-based Auto TVM search. Specifically, the specific process of template search used by TVM is as follows: (1) The user customizes a template, which includes the order of loops, data block division, whether to vectorize, etc.; (2) The search parameters are defined in the template, such as tile size (data block size); (3) Based on a given template and parameters, the framework constructs a search space, and searches for parameters in this space according to a certain strategy. For example, a specific strategy includes using a cost model (such as using a neural network to fit the current cost) to calculate the computational cost of this group of parameters, so that under the guidance of the cost model, the search strategy can search for a set of parameters with the most efficient computation. The current template search method has been widely used. For example, there are more than 15,000 lines of template code in the TVM code base.

[0120] An important tool for optimizing convolution is affine transformation, which essentially changes the access order and access interval of the loop, so as to reduce data dependence as much as possible and maximize parallel computing during the access process. Figure 5 As shown, for an original two-layer for loop, when i=2 and N=3, the inner loop is A[2,1]=Function(A[2,0],A[1,1])A[2,2]=Function(A[2,1],A[1,2]). If the loop is multi-threaded for parallel computing, at the same time point, the result of A[2,2] depends on A[2,1], so it cannot be calculated in parallel.

[0121] Affine transformation: Let y = j, x = i + j, then the overall cycle becomes Figure 6 See Figure 6, when x=3, N=3, the inner loop becomes A[2,1]=Function(A[2,0],A[1,1]),A[1,2]=Function(A[1,1],A[0,2]). At this time, there is no data dependency between A[2,1] and A[1,2], so they can be calculated in parallel at the same time. By reducing data dependency and increasing parallelism, the affine transformation supported by polyhedron technology can accelerate the calculation of neural networks.

[0122] However, the existing TVM template-based search algorithm essentially searches on a specific hardware architecture through enumeration, which results in a very large search space and low search efficiency. The costmodel in the search strategy is based on the actual execution results on the hardware, and the running time on the hardware may be very long, which results in a very long search time. Template-based search essentially enumerates the loop sequence and the size of the loop data block, which has limitations in describing the optimization method of the loop. For example, for the x86 CPU architecture, in order to maximize the data locality between multiple layers of cache (three-layer cache L1, L2, L3), the affine transformation in the polyhedron technology can maximize the locality and parallelism of the data blocks accessed between different data points by scaling the coordinate axis. Some papers have shown that this radial change plays a very important role in accelerating convolution and making full use of hardware characteristics. However, the traditional TVM-based search method cannot describe this affine transformation.

[0123] Step 104: Obtain a code template based on the polyhedron technology and the memory size of the terminal device.

[0124] For the quantized neural network, in order to deploy it on hardware, each framework will generate code that describes the data calculation method of each layer of the neural network. Different code description methods (such as parameters of loops in the code, etc.) will affect the efficiency of accessing memory and computing units. In order to improve the computing efficiency of quantized data on hardware, the present invention proposes a template code optimization and generation scheme based on polyhedron technology. Through this scheme, the present invention can search for the optimal values ​​of the parameters in the code template, thereby generating a code that can fully utilize the hardware and has the highest computing efficiency.

[0125] After the quantization strategy of steps 102 and 103 is implemented, in order to design a set of computing codes that can fully utilize the hardware, based on the polyhedron technology, the present invention designs a set of code templates for accelerating neural network calculations, in which the overall order of the loop is determined, but the parameter TILE_SIZE (data block size) in the loop is uncertain. The TILE_SIZE in the template is a search parameter. TILE_SIZE affects the locality between data, the dependency between data, and the parallelism of calculation. Generally speaking, the better the locality between data, the lower the dependency, the higher the parallelism, and the better the efficiency of calculation. When the TILE_SIZE is smaller, each access is equivalent to several rare points, which can reduce the dependency between data; when the TILE_SIZE size matches the data block of the underlying cacheline, the locality of the data can be better utilized. Therefore, different underlying architectures will have different optimal TILE_SIZE sizes.

[0126] Figure 7 A schematic diagram of a code template provided by an embodiment of the present invention. Figure 7 , the present invention takes the convolution calculation code involving AFP-encoded data as an example to illustrate the design of the template. The convolution of the neural network has two four-dimensional input data encoded using AFP: input[N,C,H,W](input) and weight[F,C,KH,KW](weight). N represents the batch of input data, C represents the channel of input data, H represents the length of input data, W represents the width of input data, F represents the convolution kernel of weight, KH represents the length of convolution kernel, and KW represents the width of convolution kernel. The output is output[N,F,OH,OW], OH represents the length of output, and OW represents the width of output. The code template describes the order of accessing data for convolution calculation: oj->f_tile->…->ifm, the data block size TILE_SIZE of access data, and the calculation rules (Line 17).

[0127] Step 105: Determine optimal parameters of the code template according to the memory size of the terminal device. The optimal parameters include data block size.

[0128] After the code template is determined, the present invention needs to determine the optimal TILE_SIZE size according to the underlying hardware. The specific processing flow is as follows.

[0129] The overall input is the above code template.

[0130] The step 105 determines the optimal parameters of the code template according to the memory size of the terminal device, and specifically includes:

[0131] Step 5.1: Determine the candidate set of the search space according to the memory size of the end-side device.

[0132] The present invention determines the sizes of several TILE_SIZEs according to the memory size of the underlying hardware, and these several TILE_SIZEs constitute a candidate set of the search space.

[0133] Step 5.2: Using the integer set library analyzed based on the polyhedron technology, taking the code template as input, and outputting a scheduling tree representing the computational relationship.

[0134] Based on given hardware (such as end-side CPU, GPU, etc.), in order to determine the optimal TILE_SIZE size, the present invention uses an integer set library (ISL) based on polyhedron technology analysis. ISL takes the above code template as input, analyzes the data dependency in the code, uses the specific syntax and semantics in polyhedron analysis, and outputs a scheduling tree representing the computational relationship. Therefore, given different TILE_SIZE, ISL will generate different scheduling trees.

[0135] Step 5.3: Based on different scheduling trees, a two-step search strategy is adopted to determine the optimal parameters in the candidate set in the search space.

[0136] Based on different scheduling trees, in order to determine the optimal TILE_SIZE, the present invention proposes a two-step search strategy. In the first step, the present invention uses the size of the data block that needs to be accessed to complete a complete calculation in each innermost loop in the scheduling tree as the screening threshold. The present invention queries the size of the data blocks in the scheduling tree in turn, takes the data blocks close to the size of the underlying memory bar as the candidate set of the optimal solution, and retains the scheduling trees corresponding to these candidate sets. In the second step, according to the rules described by the scheduling tree, rewrite the above code (for example, assign a value to TILE_SIZE, scale the coordinate axis, etc.), and then deploy these codes to run on actual hardware. Using the actual running time as the evaluation criterion, select the code with the shortest running time as the optimal generated code.

[0137] The output is the final optimized code, that is, the executable code of the neural network after dynamic quantization encoding.

[0138] Step 106: Use a code template with the optimal parameters to describe the data calculation method of each layer of the dynamically quantized encoded neural network, and generate an executable code for the dynamically quantized encoded neural network.

[0139] Based on the designed code template, the present invention can obtain the executable code of the whole network encoded in AFP format which can be efficiently run on hardware. Therefore, based on AFP format and optimized executable code, the present invention can accelerate the whole calculation.

[0140] Step 107: Acquire image data encoded in the AFP format. The image data includes face image data and object image data.

[0141] The different application scenarios of neural networks are determined by their structure (number of layers, etc.). Therefore, different neural networks can be applied in different scenarios, such as image recognition, object detection, etc.

[0142] The step 107 of acquiring the image data encoded in the AFP format specifically includes:

[0143] Step 7.1: Acquire an image received by the terminal device; the image includes a face image or an object image;

[0144] Step 7.2: Encode the image in the AFP format to generate image data encoded in the AFP format.

[0145] Step 108: deploy the executable code to the terminal device to recognize the image data and obtain an image recognition result.

[0146] The end-side device receives an image of a face or an object and encodes the image into image data encoded in the FP32 data format. At the same time, the end-side device selects a neural network with a specific structure encoded in a traditional data format, that is, a collection of multiple layers of data encoded in FP32. These two parts of data enter the processing flow of the present invention as the original input.

[0147] In order to adapt the data to speed up the calculation at the same time, the present invention firstly expresses the above two parts of data in AFP format. In order to determine the encoding bit width (n exp , n max ) and the set of offset values ​​k (n exp , n man , k) the actual size, using the algorithm in step 102. After algorithm exploration, all parameters determine the bit width, and the data of the above two parts are represented as data in AFP format.

[0148] In order to deploy the above process to the end-side device, the end-side device will generate executable code. The executable code represents the calculation rules of each layer, and the calculation of each layer is to perform corresponding calculations on the above two parts of data (such as multiplication, addition, division, etc.). The writing rules of the code directly affect the final efficiency. Therefore, the code for calculating the data encoded in the AFP format is used as input, and the optimized code is obtained through the search process of steps 105-106.

[0149] The optimized executable code encoded in AFP format is deployed to the corresponding end-side device for calculation. For the input face or object image data, after the above calculation, the image recognition result (0 / 1) indicating whether the match is successful is output, 0 indicates a failed match, and 1 indicates a successful match.

[0150] Based on the image recognition method based on adaptive data quantization and polyhedron template provided by the present invention, the present invention also provides an image recognition system based on adaptive data quantization and polyhedron template. Figure 8 A structural diagram of an image recognition system based on adaptive data quantization and polyhedral template provided by an embodiment of the present invention, see Figure 8 , the system comprising:

[0151] The traditional encoding neural network acquisition module 801 is used to acquire a neural network encoded in a traditional data format selected by the terminal device; the traditional data format includes an FP32 data format;

[0152] A Bayesian optimization search module 802 is used to determine an optimal value set of a floating point representation AFP of adaptive data according to the neural network encoded in the traditional data format by using a search algorithm based on Bayesian optimization; the optimal value set includes an exponent bit width, a mantissa bit width and an offset value corresponding to each layer in the neural network;

[0153] An AFP quantization encoding module 803 is used to encode the neural network in the AFP format according to the optimal value set to generate a dynamically quantized encoded neural network;

[0154] A code template acquisition module 804 is used to acquire a code template based on the polyhedron technology and the memory size of the terminal device;

[0155] An optimal parameter determination module 805, configured to determine the optimal parameters of the code template according to the memory size of the terminal device; the optimal parameters include a data block size;

[0156] An executable code generation module 806 is used to use a code template with the optimal parameters to describe the data calculation method of each layer of the neural network after the dynamic quantization encoding, and generate an executable code of the neural network after the dynamic quantization encoding;

[0157] An image data acquisition module 807 is used to acquire image data encoded in the AFP format; the image data includes face image data and object image data;

[0158] The image recognition module 808 is used to deploy the executable code to the terminal device to recognize the image data and obtain an image recognition result.

[0159] The Bayesian optimization search module 802 specifically includes:

[0160] A candidate set generation unit, used to generate a plurality of candidate sets of possible values ​​according to a Bayesian optimization algorithm;

[0161] The optimal value set selection unit is used to select the optimal value set from the candidate set according to a preset calculation function.

[0162] The optimal parameter determination module 805 specifically includes:

[0163] A search space determination unit, configured to determine a candidate set of the search space according to the memory size of the terminal device;

[0164] A scheduling tree output unit, used to use the integer set library analyzed based on polyhedron technology, take the code template as input, and output a scheduling tree representing the calculation relationship;

[0165] The optimal parameter determination unit is used to determine the optimal parameter in the candidate set of the search space by adopting a two-step search strategy based on different scheduling trees.

[0166] The image data acquisition module 807 specifically includes:

[0167] An image acquisition unit, configured to acquire an image received by the terminal device; the image includes a face image or an object image;

[0168] The image AFP encoding unit is used to encode the image in the AFP format to generate image data encoded in the AFP format.

[0169] In order to solve the waste of accuracy and representation range of traditional quantized data formats on data sets, the present invention proposes a solution that can quantize in an adaptive data format. At the same time, in order to solve the limitations of the description method of traditional TVM-based templates, the mismatch between search effects and search strategies, the present invention proposes a search space design and search strategy design based on polyhedron technology. Finally, the present invention accelerates the computational efficiency of neural networks, improves computational accuracy, and saves equipment resources (energy consumption, etc.) through quantization technology and polyhedron-based search strategies.

[0170] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0171] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. An image recognition method based on adaptive data quantization and polyhedral template, characterized in that: include: Obtaining a neural network encoded in a traditional data format selected by the end-side device; the traditional data format includes an FP32 data format; According to the neural network encoded in the traditional data format, an optimal value set of a floating point representation AFP of the adaptive data is determined by using a search algorithm based on Bayesian optimization; the optimal value set includes an exponent bit width, a mantissa bit width and an offset value corresponding to each layer in the neural network; For each layer of data in the neural network, the floating point representation of the adaptive data is b AFP 1 sign bit n exp bit exponent bit and n man The mantissa is composed of digits; exp and n man The bit width is adjusted according to the actual data of each layer. The specific data encoding format is as follows: For a floating point number encoded as a binary string of 0s and 1s, the value g(b AFP ) is calculated according to the following formula: Where k represents the offset value; According to the optimal value set, encoding the neural network in the AFP format to generate a dynamically quantized encoded neural network; Acquire a code template based on polyhedron technology and a memory size of the terminal device; Determining optimal parameters of the code template according to the memory size of the terminal device; the optimal parameters include data block size; Using a code template with the optimal parameters to describe the data calculation method of each layer of the neural network after the dynamic quantization encoding, and generating an executable code of the neural network after the dynamic quantization encoding; Acquire image data encoded in the AFP format; the image data includes face image data and object image data; The executable code is deployed on the terminal device to recognize the image data and obtain an image recognition result.

2. The method according to claim 1, characterized in that The method of determining the optimal value set of AFPs based on the neural network encoded in the traditional data format by using a search algorithm based on Bayesian optimization specifically includes: Generate a candidate set of multiple possible values ​​according to a Bayesian optimization algorithm; The optimal value set is selected from the candidate set according to a preset calculation function.

3. The method according to claim 1, characterized in that The determining the optimal parameters of the code template according to the memory size of the terminal device specifically includes: Determine a candidate set of the search space according to the memory size of the terminal device; Using an integer set library analyzed based on polyhedron technology, taking the code template as input, and outputting a scheduling tree representing a computational relationship; Based on different scheduling trees, a two-step search strategy is adopted to determine the optimal parameters in the candidate set of the search space.

4. The method according to claim 1, characterized in that The acquiring of the image data encoded in the AFP format specifically comprises: Collecting an image received by the terminal device; the image includes a face image or an object image; The image is encoded in the AFP format to generate image data encoded in the AFP format.

5. An image recognition system based on adaptive data quantization and polyhedral templates, characterized in that: include: A traditional encoding neural network acquisition module, used to acquire a neural network encoded in a traditional data format selected by a terminal device; the traditional data format includes an FP32 data format; A Bayesian optimization search module is used to determine an optimal value set of a floating point expression form AFP of adaptive data according to the neural network encoded in the traditional data format by using a search algorithm based on Bayesian optimization; the optimal value set includes an exponent bit width, a mantissa bit width and an offset value corresponding to each layer in the neural network; For each layer of data in the neural network, the floating point representation of the adaptive data is b AFP 1 sign bit n exp bit exponent bit and n man The mantissa is composed of digits; exp and n man The bit width is adjusted according to the actual data of each layer. The specific data encoding format is as follows: For a floating point number encoded as a binary string of 0s and 1s, the value g(b AFP ) is calculated according to the following formula: Where k represents the offset value; An AFP quantization encoding module is used to encode the neural network in the AFP format according to the optimal value set to generate a neural network after dynamic quantization encoding; A code template acquisition module, used to acquire a code template based on polyhedron technology and a memory size of the terminal device; An optimal parameter determination module, used to determine the optimal parameters of the code template according to the memory size of the terminal device; the optimal parameters include the data block size; An executable code generation module, used to describe the data calculation method of each layer of the neural network after the dynamic quantization encoding by using a code template with the optimal parameters, and generate an executable code of the neural network after the dynamic quantization encoding; An image data acquisition module, used to acquire image data encoded in the AFP format; the image data includes face image data and object image data; The image recognition module is used to deploy the executable code to the terminal device to recognize the image data and obtain an image recognition result.

6. The system according to claim 5, characterized in that The Bayesian optimization search module specifically includes: A candidate set generation unit, used to generate a plurality of candidate sets of possible values ​​according to a Bayesian optimization algorithm; The optimal value set selection unit is used to select the optimal value set from the candidate set according to a preset calculation function.

7. The system according to claim 5, characterized in that The optimal parameter determination module specifically includes: A search space determination unit, configured to determine a candidate set of the search space according to the memory size of the terminal device; A scheduling tree output unit, used to use the integer set library analyzed based on polyhedron technology, take the code template as input, and output a scheduling tree representing the calculation relationship; The optimal parameter determination unit is used to determine the optimal parameter in the candidate set of the search space by adopting a two-step search strategy based on different scheduling trees.

8. The system according to claim 5, characterized in that The image data acquisition module specifically includes: An image acquisition unit, configured to acquire an image received by the terminal device; the image includes a face image or an object image; The image AFP encoding unit is used to encode the image in the AFP format to generate image data encoded in the AFP format.

Images