Instruction generation method and device, data processing method and device

Abstract

Disclosed are an instruction generation method and device, and a data processing method and device. The instruction generation method includes: determining the model features of at least one neural network model sample; determining the compression method of at least one neural network model sample based on the model features of at least one neural network model sample; and generating a compression instruction based on the model features and compression method of at least one neural network model sample. Since the model features can characterize the model structure characteristics of the neural network model, and the model structure characteristics can determine the data characteristics of the data to be migrated output by the neural network model, and different compression methods also have their own compression characteristics, the present disclosure can make the determined compression method and the neural network model sample more matched, and thus can make the generated compression instruction fully take into account the characteristics of the model features and the compression method.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to an instruction generation method and device, and a data processing method and device. Background Art

[0002] As we all know, Neural Network (NN) is a very powerful machine learning algorithm, such as Convolutional Neural Network (CNN). In recent years, neural networks have received increasing attention in the fields of pattern recognition and intelligent control. However, when performing related intelligent calculations based on neural networks, the amount of data to be migrated is very large, which leads to low chip access efficiency and affects the execution efficiency of the network model. Summary of the invention

[0003] In order to solve the above technical problems, the present disclosure is proposed. The embodiments of the present disclosure provide an instruction generation method and device, and a data processing method and device.

[0004] On the one hand, an embodiment of the present disclosure provides an instruction generation method, which includes: determining model characteristics of at least one neural network model sample; determining a compression method of at least one neural network model sample based on the model characteristics of at least one neural network model sample; generating a compression instruction based on the model characteristics and compression method of at least one neural network model sample, the compression instruction being used to perform a compression operation on the data to be migrated output by the neural network model based on the model characteristics of the neural network model.

[0005] On the other hand, an embodiment of the present disclosure provides a data processing method, which includes: determining model characteristics of a neural network model to be processed; based on the model characteristics of the neural network model to be processed and compression instructions, performing a compression operation on the data to be migrated output by the network model to be processed to obtain compressed data, wherein the compression instructions are generated based on the instruction generation method mentioned above.

[0006] On the other hand, an embodiment of the present disclosure provides an instruction generating device, which includes: a model feature determination module, used to determine the model features of at least one neural network model sample; a compression method determination module, used to determine the compression method of at least one neural network model sample based on the model features of at least one neural network model sample; a compression instruction generation module, used to generate a compression instruction based on the model features and compression method of at least one neural network model sample, the compression instruction being used to perform a compression operation on the data to be migrated output by the neural network model based on the model features of the neural network model.

[0007] On the other hand, an embodiment of the present disclosure provides a data processing device, which includes: a determination module for determining model characteristics of a neural network model to be processed; a compression module for performing a compression operation on the data to be migrated output by the neural network model to be processed based on the model characteristics of the neural network model to be processed and compression instructions to obtain compressed data, wherein the compression instructions are generated based on the instruction generation method mentioned above.

[0008] On the other hand, an embodiment of the present disclosure provides a computer-readable storage medium, which stores a computer program for executing the above-mentioned instruction generation method and / or data processing method.

[0009] On the other hand, an embodiment of the present disclosure provides an electronic device, comprising: a processor and a memory for storing instructions executable by the processor, wherein the processor is used to execute the above-mentioned instruction generation method and / or data processing method.

[0010] Since the model features can characterize the model structure characteristics of the neural network model, and the model structure characteristics can determine the data characteristics of the data to be migrated output by the neural network model, and different compression methods also have their own compression characteristics, therefore, the compression method of the neural network model sample determined based on the model features of the neural network model sample mentioned in the embodiment of the present disclosure can make the determined compression method and the neural network model sample more matched, and then the method of generating compression instructions based on the model features and compression methods of the neural network model sample can make the generated compression instructions fully take into account the characteristics of the model features and the compression methods. If the instruction generation method provided by the embodiment of the present disclosure is applied to the execution process of the neural network model, the purpose of automatically selecting a matching compression method for the data to be migrated output by the neural network model according to the actual characteristics of the neural network model can be achieved with the help of the embodiment of the present disclosure, thereby achieving the purpose of reducing the load and delay of data transmission on the bus, improving chip access efficiency, and improving the execution efficiency of the neural network model. In other words, based on the instruction generation method mentioned in the embodiment of the present disclosure, the purpose of adaptively compressing the data to be migrated according to the data characteristics of the neural network model can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other purposes, features and advantages of the present disclosure will become more apparent by describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps.

[0012] Figure 1The figure shows a scene to which the present disclosure is applicable.

[0013] Figure 2 Another scene diagram to which the present disclosure is applicable is shown.

[0014] Figure 3 FIG. 1 is a flow chart of an instruction generation method provided by an exemplary embodiment of the present disclosure.

[0015] Figure 4 The figure is a schematic diagram of a flow chart for determining the model characteristics of at least one neural network model sample according to an exemplary embodiment of the present disclosure.

[0016] Figure 5 FIG. 1 is a flow chart of an instruction generation method provided by another exemplary embodiment of the present disclosure.

[0017] Figure 6 FIG. 1 is a schematic diagram showing the principle of a Huffman encoding method provided by an exemplary embodiment of the present disclosure.

[0018] Figure 7 The figure shows a coding logic diagram of a Huffman coding method provided by an exemplary embodiment of the present disclosure.

[0019] Figure 8 FIG. 4 is a flow chart of an instruction generation method provided by another exemplary embodiment of the present disclosure.

[0020] Fig. 9 FIG. 1 is a flow chart of an instruction generation method provided by yet another exemplary embodiment of the present disclosure.

[0021] Fig.10 The figure shows a coding logic diagram of a zero-ary compression coding method provided by an exemplary embodiment of the present disclosure.

[0022] Fig.11 Shown is a schematic diagram of a hardware execution scenario provided by an exemplary embodiment of the present disclosure.

[0023] Fig.12 Shown is a flowchart of a data processing method provided by an exemplary embodiment of the present disclosure.

[0024] Fig.13 Shown is a flowchart of a data processing method provided by another exemplary embodiment of the present disclosure.

[0025] Fig.14 Shown is a schematic diagram of the structure of an instruction generation device provided by an exemplary embodiment of the present disclosure.

[0026] Fig.15 FIG. 1 is a schematic diagram of the structure of a data processing device provided by an exemplary embodiment of the present disclosure.

[0027] Fig.16 Shown is a schematic structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] Below, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described here.

[0029] Application Overview

[0030] The integration of artificial intelligence (AI) and the Internet of Things will inevitably lead to a spurt in data growth. At present, cloud computing is usually used to alleviate the storage and management problems brought about by the growing data, but the current growth rate of network bandwidth is far less than the growth rate of data. In addition, the complex network environment makes it difficult to make a breakthrough in network latency. Take self-driving cars as an example. High-speed cars need to respond within milliseconds. Once the system response time increases due to data transmission, network and other problems, it will cause unpredictable serious consequences.

[0031] Especially for edge AI involving neural networks, a large amount of data migration is involved in the edge AI computing process. The main form of migration is to move data between the double data rate synchronous dynamic random access memory (DDR) and the static random access memory (SRAM) through the bus. It can be seen that the data migration speed largely determines the chip access efficiency, which in turn affects the speed of executing AI calculations, that is, the execution efficiency of the final neural network model.

[0032] In view of the above technical problems, the basic concept of the present disclosure is to propose an instruction generation method and device, and a data processing method and device. The instruction generation method includes: determining the model characteristics of at least one neural network model sample; determining the compression method of at least one neural network model sample based on the model characteristics of at least one neural network model sample; generating a compression instruction based on the model characteristics and compression method of at least one neural network model sample, and the compression instruction is used to compress the data to be migrated output by the neural network model based on the model characteristics of the neural network model.

[0033] Since the model features can characterize the model structure characteristics of the neural network model, and the model structure characteristics can determine the data characteristics of the data to be migrated output by the neural network model, and different compression methods also have their own compression characteristics, therefore, the compression method of the neural network model sample determined based on the model features of the neural network model sample mentioned in the embodiment of the present disclosure can make the determined compression method and the neural network model sample more matched, and then the method of generating compression instructions based on the model features and compression methods of the neural network model sample can make the generated compression instructions fully take into account the characteristics of the model features and the compression methods. If the instruction generation method provided by the embodiment of the present disclosure is applied to the execution process of the neural network model, the purpose of automatically selecting a matching compression method for the data to be migrated output by the neural network model according to the actual characteristics of the neural network model can be achieved with the help of the embodiment of the present disclosure, thereby achieving the purpose of reducing the load and delay of data transmission on the bus, improving chip access efficiency, and improving the execution efficiency of the neural network model. In other words, based on the instruction generation method mentioned in the embodiment of the present disclosure, the purpose of adaptively compressing the data to be migrated according to the data characteristics of the neural network model can be achieved.

[0034] After introducing the basic principles of the present disclosure, various non-limiting embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0035] Exemplary Systems

[0036] Figure 1 FIG. 1 is a scene diagram to which the present disclosure is applicable. Figure 1 As shown, the application scenarios of the present disclosure are application scenarios involving images. Specifically, Figure 1 The scene shown includes an image acquisition device 10 and an image recognition device 30 connected to the image acquisition device 10. The image recognition device 30 is used to perform image recognition using an image recognition model 31. Exemplarily, the image processing model 21 is a neural network model that can perform relevant processing (such as denoising) on ​​an image. The image recognition model 31 is a neural network model that can recognize an image (such as target recognition).

[0037] If an image recognition scenario is to be implemented, the image acquisition device 10 is used to acquire an image to be recognized, the image recognition device 30 obtains the image to be recognized acquired by the image acquisition device 10, and uses the image recognition model 31 to perform recognition processing on the image to be recognized. In addition, the image acquisition device 10 can also determine a compression method based on a pre-generated compression instruction and a model feature of a neural network model (such as an image processing model 21 or an image recognition model 31), and use the compression method to compress the data to be migrated output by the neural network model (such as the image recognition model 31).

[0038] The compression instruction is generated by: determining the model characteristics of at least one neural network model sample; determining the compression method of at least one neural network model sample based on the model characteristics of at least one neural network model sample; and generating the compression instruction based on the model characteristics and compression method of at least one neural network model sample. The compression instruction may include a rule for determining the compression method, i.e., a rule for characterizing the compression methods corresponding to different model characteristics.

[0039] Since the application scenarios involving images mentioned in the embodiments of the present disclosure are directly implemented using hardware devices (such as image recognition devices 30), there is no need to perform data transmission operations with devices such as servers, so the real-time nature of image recognition operations can be fully guaranteed. In addition, with the help of pre-generated compression instructions, the purpose of adaptively compressing the data to be migrated can be achieved according to the data characteristics of the neural network model (such as the image recognition model).

[0040] Exemplarily, the recognition image mentioned above is an image within the driving range of the movable device, so as to better assist the driving operation of the movable device based on the image recognition model 31. The movable device includes but is not limited to vehicles, drones, logistics carts, sweeping robots, autonomous humanoid robots and other devices with mobile capabilities. In addition, it can be understood that Figure 1 The image processing device 20 and the image recognition device 30 shown do not necessarily exist at the same time, and can be adjusted according to actual conditions.

[0041] Figure 2 FIG. 2 is another scene diagram applicable to the present disclosure. Figure 2 As shown, the scenario to which the present disclosure is applicable is an application scenario involving audio. Specifically, Figure 2 The scene shown includes a server 40 and an audio and video player 50 connected to the server 40. The server 40 includes an audio processing model 41. Exemplarily, the audio processing model 41 is a neural network model capable of performing relevant processing (such as denoising) on ​​audio.

[0042] For example, in actual application, the server 40 is used to process the audio to be processed, and then the audio and video player 50 plays the audio processed by the audio processing model 41. Figure 1 Description of the scenario shown. That is, Figure 2 The scenario shown implements the audio processing with the aid of a server 40 .

[0043] It should be understood that the instruction generation method and data processing method provided in the following embodiments of the present disclosure are not limited to the image scenes and audio scenes mentioned above, but fall within the scope of application of the embodiments of the present disclosure as long as the scenes involve neural network models.

[0044] Exemplary Methods

[0045] Figure 3 FIG. 1 is a flow chart of an instruction generation method provided by an exemplary embodiment of the present disclosure. Figure 3 As shown, the instruction generation method provided by the embodiment of the present disclosure includes the following steps.

[0046] Step S100, determining the model characteristics of at least one neural network model sample.

[0047] Exemplarily, the model feature refers to a feature that can characterize the model structure characteristics of the neural network model. For example, the model feature includes the parameter standard deviation feature of the convolution kernel in the convolution layer, and for another example, the model feature includes the activation function feature included in the activation function layer.

[0048] Exemplarily, a neural network model sample is a neural network model that can be used as a sample, in essence, a neural network model that is used as an example. The disclosed embodiments do not limit the specific model structure of the neural network model that can be used as a sample. For example, a neural network model including a convolutional layer can be used as a neural network model sample. For another example, a neural network model including an activation function layer can be used as a neural network model sample.

[0049] Step S200: Determine a compression method for at least one neural network model sample based on the model characteristics of at least one neural network model sample.

[0050] Exemplarily, the compression method includes but is not limited to Huffman coding, zero-ary compression coding, etc.

[0051] Step S300, generating a compression instruction based on the model characteristics and compression method of at least one neural network model sample.

[0052] That is, a compression instruction is generated based on the M model features and the compression methods corresponding to the M model features, where M is a positive integer. The compression instruction is used to compress the data to be migrated output by the neural network model based on the model features of the neural network model.

[0053] Exemplarily, the data to be migrated is data output by a neural network model sample and needs to be transferred between DDR and SRAM through a bus, such as a feature map.

[0054] Exemplarily, the compression instruction includes: an instruction for performing a compression operation in a zero-ary compression encoding manner on Huffman tree data generated based on the Huffman encoding manner. It should be noted that the instruction is only an example, and the present application is not limited thereto.

[0055] Exemplarily, in actual application, the model characteristics of at least one neural network model sample are first determined, and then based on the model characteristics of at least one neural network model sample, the compression method of at least one neural network model sample is determined, and then based on the model characteristics and compression method of at least one neural network model sample, a compression instruction is generated.

[0056] Since the model features can characterize the model structure characteristics of the neural network model, and the model structure characteristics can determine the data characteristics of the data to be migrated output by the neural network model, and different compression methods also have their own compression characteristics, the method of determining the compression method of the neural network model sample based on the model features of the neural network model sample mentioned in the embodiment of the present disclosure, and then generating the compression instruction based on the model features and compression method of the neural network model sample, can make the generated compression instruction fully take into account the characteristics of the model features and the compression method. If the instruction generation method provided by the embodiment of the present disclosure is applied to the execution process of the neural network model, the purpose of automatically selecting a matching compression method for the data to be migrated output by the neural network model according to the actual characteristics of the neural network model can be achieved with the help of the embodiment of the present disclosure, thereby achieving the purpose of reducing the load and delay of the data to be migrated transmitted on the bus, improving the chip access efficiency, and improving the execution efficiency of the neural network model. In other words, based on the instruction generation method mentioned in the embodiment of the present disclosure, the purpose of adaptively compressing the data to be migrated according to the data characteristics of the neural network model can be achieved.

[0057] In some embodiments, the compression instruction includes: an instruction to store compression mode information that can characterize the compression mode corresponding to the neural network model sample into the compressed data packet to be migrated. Such a setting can facilitate subsequent decoding operations.

[0058] Figure 4 The figure is a flow chart of determining the model characteristics of at least one neural network model sample according to an exemplary embodiment of the present disclosure. Figure 3 The present disclosure is extended based on the embodiment shown Figure 4 The embodiment shown is described below in detail. Figure 4 The embodiment shown and Figure 3 The differences and similarities of the illustrated embodiments are not described in detail.

[0059] like Figure 4 As shown, in the embodiment of the present disclosure, the step of determining the model characteristics of at least one neural network model sample includes the following steps.

[0060] Step S110, determining the hierarchical structure of at least one neural network model sample.

[0061] It can be understood that the hierarchical structure refers to the hierarchical structure included in the neural network model sample. For example, the layers of data input layer, excitation layer and convolution layer and the data transmission relationship between these layers constitute a hierarchical structure (i.e., a hierarchical structure).

[0062] Step S120: determining the model features of at least one neural network model sample based on the hierarchical structure of at least one neural network model sample.

[0063] Since the hierarchical structure of the neural network model sample can fully characterize the data characteristics of the data to be migrated output by the neural network model sample, and the hierarchical structure information of the neural network model sample is extremely easy to obtain, the method of determining model features using the hierarchical structure mentioned in the embodiment of the present disclosure can greatly improve the speed of determining model features and greatly reduce the difficulty of determination.

[0064] Combine the following Figure 5 An example is given to illustrate how to determine the model features of a neural network model sample based on the hierarchical structure of the neural network model sample, and then determine the specific implementation method of the compression method based on the model features.

[0065] Figure 5 FIG. 1 is a flow chart of an instruction generation method provided by another exemplary embodiment of the present disclosure. Figure 4 The present disclosure is extended based on the embodiment shown Figure 5 The embodiment shown is described below in detail. Figure 5 The embodiment shown and Figure 4 The differences and similarities of the illustrated embodiments are not described in detail.

[0066] like Figure 5 As shown, in an embodiment of the present disclosure, based on the hierarchical structure of at least one neural network model sample, the step of determining the model features of at least one neural network model sample includes the following steps.

[0067] Step S121: for each neural network model sample in at least one neural network model sample, determine whether the neural network model sample includes a convolutional layer based on the hierarchical structure of the neural network model sample.

[0068] It can be understood that a convolutional layer includes at least one convolution unit. It can be understood that the purpose of the convolution operation is to extract different features of the input.

[0069] Exemplarily, in step S121, if the judgment result is yes, that is, based on the hierarchical structure of the neural network model sample, it is determined that the neural network model sample includes a convolutional layer, then step S122 is executed; if the judgment result is no, that is, based on the hierarchical structure of the neural network model sample, it is determined that the neural network model sample does not include a convolutional layer, then the relevant method flow can be terminated.

[0070] Step S122, using the parameter standard deviation of the convolution kernel in the convolution layer to characterize the model characteristics of the neural network model sample.

[0071] It should be noted that since the parameter standard deviation of the convolution kernel (also known as the standard deviation of the parameter) reflects the degree of dispersion of the parameter from the mean, it can characterize the average degree of the parameters of the convolution kernel. Correspondingly, the smaller the parameter standard deviation, the more average the parameters of the convolution kernel. That is, the parameter standard deviation is inversely proportional to the average degree.

[0072] Furthermore, in an embodiment of the present disclosure, based on the model characteristics of at least one neural network model sample, the steps of determining the compression method of at least one neural network model sample include the following steps.

[0073] Step S210, calculating the standard deviation of the parameters of the convolution kernel in the convolution layer.

[0074] Step S220, determining whether the parameter standard deviation meets a preset condition.

[0075] Exemplarily, in step S220, if the judgment result is yes, that is, the parameter standard deviation meets the preset condition, step S230 is executed, and if the judgment result is no, that is, the parameter standard deviation does not meet the preset condition, the relevant method flow can be terminated. For example, the preset condition is less than the preset standard deviation threshold. Then, if the parameter standard deviation is less than the preset standard deviation threshold, step S230 is executed, and if the parameter standard deviation is greater than or equal to the preset standard deviation threshold, the relevant method flow can be terminated.

[0076] Step S230, determining that the compression method corresponding to the neural network model sample is the Huffman coding method.

[0077] Exemplarily, in actual application, the hierarchical structure of at least one neural network model sample is first determined, and then for each neural network model sample in the at least one neural network model sample, it is determined whether the neural network model sample includes a convolution layer based on the hierarchical structure of the neural network model sample. If it is determined that the neural network model sample includes a convolution layer, the parameter standard deviation of the convolution kernel in the convolution layer is used to characterize the model characteristics of the neural network model sample, and then the parameter standard deviation of the convolution kernel in the convolution layer is calculated to determine whether the parameter standard deviation meets the preset conditions. If the parameter standard deviation meets the preset conditions, it is determined that the compression method corresponding to the neural network model sample is the Huffman coding method.

[0078] The data characteristics of data compressed based on Huffman coding include: in a data set, data with higher frequency of occurrence before coding will have shorter codes, and data with lower frequency of occurrence before coding will have longer codes. Correspondingly, in the disclosed embodiment, since the parameter standard deviation can characterize the average degree of the parameters of the convolution kernel, and the parts with better average degree are likely to have the same value, a shorter code will be obtained after compressing the parts with better average degree, thereby achieving a compression effect. Based on this, the disclosed embodiment mentioned in the method of determining whether the compression method corresponding to the neural network model sample is Huffman coding based on the parameter standard deviation of the convolution kernel can make full use of the advantages of Huffman coding, thereby ultimately generating compression instructions with good compression effect.

[0079] To further clarify the Huffman coding method, the following Figure 6 and Figure 7 Explain the principle and encoding logic of Huffman coding.

[0080] Figure 6 FIG. 1 is a schematic diagram showing the principle of a Huffman encoding method provided by an exemplary embodiment of the present disclosure. Figure 7 FIG. 1 is a schematic diagram of the encoding logic of the Huffman encoding method provided by an exemplary embodiment of the present disclosure. Figure 6 and Figure 7 As shown in the figure, assume that the feature map output by the neural network model sample is 16 bytes, and each byte represents a pixel in the feature map. Assume that these 16 bytes are represented as aaaaaaaabbbbccdd, and each letter represents a byte. The same letters represent the same bytes, and different letters represent different bytes. Then, based on the 16 bytes, the frequency of each byte can be counted, as follows: Figure 6 As shown in the figure, the frequencies with large frequencies are arranged on the top and the frequencies with small frequencies are arranged on the bottom. Then the frequencies are synthesized step by step according to the method in the figure.

[0081] (1) Start from the bottom and assign 0 and 1 to the bottom two frequencies.

[0082] (2) The two frequencies at the bottom are synthesized into one, that is, the frequencies are added, and then rearranged with the larger frequency at the top and the smaller frequency at the bottom. If the frequencies are the same, the newly synthesized one is preferably arranged at the top.

[0083] (3) Repeat steps (1) and (2) until the added frequency is 1.0.

[0084] After the synthesis is completed, the Huffman tree (i.e. Figure 6As shown in the lower left corner). It can be understood that the codebook of the Huffman tree dimension is the tool for Huffman algorithm decoding and encoding, which is represented in the algorithm in the form of a tree structure. Specifically, starting from the root and going down, the 0s and 1s encountered along the path are connected to form the corresponding code, and finally we get Figure 6 The coding table in the lower right corner.

[0085] Figure 8 FIG. 1 is a flow chart of an instruction generation method provided by another exemplary embodiment of the present disclosure. Figure 5 The present disclosure is extended based on the embodiment shown Figure 8 The embodiment shown is described below in detail. Figure 8 The embodiment shown and Figure 5 The differences and similarities of the illustrated embodiments are not described in detail.

[0086] like Figure 8 As shown, in the embodiment of the present disclosure, before determining that the compression method corresponding to the neural network model sample is the Huffman coding method (ie, step S230), the following steps are also included.

[0087] Step S221, determining the number of bytes of the data sample to be migrated output by the neural network model sample.

[0088] Step S222, determining whether the number of bytes is greater than a preset byte number threshold.

[0089] Exemplarily, in step S222, if the judgment result is yes, that is, the number of bytes is greater than the preset byte number threshold, step S230 is executed. If the judgment result is no, that is, the number of bytes is less than or equal to the preset byte number threshold, the relevant method flow can be terminated.

[0090] Exemplarily, in actual application, the hierarchical structure of at least one neural network model sample is first determined, and then for each neural network model sample in the at least one neural network model sample, based on the hierarchical structure of the neural network model sample, it is determined whether the neural network model sample includes a convolution layer; if it is determined that the neural network model sample includes a convolution layer, the parameter standard deviation of the convolution kernel in the convolution layer is used to characterize the model characteristics of the neural network model sample, and then the parameter standard deviation of the convolution kernel in the convolution layer is calculated to determine whether the parameter standard deviation meets the preset conditions; if the parameter standard deviation meets the preset conditions, the number of bytes of the data sample to be migrated output by the neural network model sample is determined, and it is determined whether the number of bytes is greater than the preset byte number threshold; if the parameter standard deviation meets the preset conditions and the number of bytes is greater than the preset byte number threshold, it is determined that the compression method corresponding to the neural network model sample is the Huffman coding method.

[0091] As mentioned above, Huffman coding generates a Huffman tree for decoding and encoding. Without the Huffman tree, decoding is impossible. Therefore, the generated Huffman tree needs to be bound to the compressed data corresponding to the data to be migrated, and operations such as transmission and / or downloading are performed together.

[0092] It can be seen that the Huffman tree will inevitably occupy a certain number of bytes. In order to prevent the situation where the data becomes more after being compressed based on the Huffman encoding method due to the existence of the Huffman tree, the embodiment of the present disclosure pre-judges the number of bytes of the data sample to be migrated before encoding, and only adopts the Huffman encoding method when the number of bytes is greater than the preset byte number threshold, thereby further improving the rationality of the generated compression instructions.

[0093] Fig. 9 FIG. 1 is a flow chart of an instruction generation method provided by another exemplary embodiment of the present disclosure. Figure 4 The present disclosure is extended based on the embodiment shown Fig. 9 The embodiment shown is described below in detail. Fig. 9 The embodiment shown and Figure 4 The differences and similarities of the illustrated embodiments are not described in detail.

[0094] like Fig. 9 As shown, in an embodiment of the present disclosure, based on the hierarchical structure of at least one neural network model sample, the step of determining the model features of at least one neural network model sample includes the following steps.

[0095] Step S123: for each neural network model sample in the at least one neural network model sample, determine whether the neural network model sample includes an activation function layer based on the hierarchical structure of the neural network model sample.

[0096] Exemplarily, in step S123, if the judgment result is yes, that is, based on the hierarchical structure of the neural network model sample, it is determined that the neural network model sample includes an activation function layer, then step S124 is executed; if the judgment result is no, that is, based on the hierarchical structure of the neural network model sample, it is determined that the neural network model sample does not include an activation function layer, then the relevant method flow can be terminated.

[0097] Step S124, using the activation function included in the activation function layer to characterize the model characteristics of the neural network model sample.

[0098] Furthermore, in an embodiment of the present disclosure, based on the model characteristics of at least one neural network model sample, the steps of determining the compression method of at least one neural network model sample include the following steps.

[0099] Step S240, determining that the compression method corresponding to the neural network model sample is a zero-ary compression encoding method based on the activation function.

[0100] Exemplarily, in actual application, the hierarchical structure of at least one neural network model sample is first determined, and then for each neural network model sample in the at least one neural network model sample, it is determined whether the neural network model sample includes an activation function layer based on the hierarchical structure of the neural network model sample. If it is determined that the neural network model sample includes an activation function layer, the activation function included in the activation function layer is used to characterize the model characteristics of the neural network model sample, and then based on the activation function, it is determined that the compression method corresponding to the neural network model sample is a zero-ary compression encoding method.

[0101] Since a large amount of zero metadata is generated when processing data based on an activation function, and transmitting a large amount of zero metadata will cause unnecessary waste of bandwidth, the embodiment of the present disclosure solves the problem of wasting bandwidth and improves data transmission and chip access efficiency.

[0102] Fig.10 FIG. 1 is a schematic diagram of the encoding logic of a zero-ary compression encoding method provided by an exemplary embodiment of the present disclosure. Fig.10 As shown, 16 bytes of data are compressed to 9 bytes. The more zero-yuan metadata there is in the data to be compressed (such as the data sample to be migrated), the greater the degree of compression. The core concept of the zero-yuan compression encoding method is to use one bit to represent one byte. Specifically, for each bit, if the bit is 0, it represents a zero-yuan byte, and if the bit is 1, it represents a non-zero-yuan byte. The non-zero-yuan bytes are stored in order, and the last byte stores the code number used to indicate that the encoding method is the zero-yuan compression encoding method, so that the receiving end can perform decoding operations.

[0103] Combine the following Fig.11 The actual hardware execution scenario of the instruction generation method mentioned in the above embodiment is further illustrated by examples.

[0104] Fig.11 FIG. 1 is a schematic diagram of a hardware execution scenario provided by an exemplary embodiment of the present disclosure. Fig.11 As shown, at least one neural network model sample is first input into the compiler, and the normal download / store (i.e., load / store) instruction and the load / store instruction with compression instruction are generated in combination with the instruction generation method mentioned in the above embodiment, wherein the compression instruction involves the Huffman encoding method and the zero-ary compression encoding method mentioned in the above embodiment. Finally, the branch processing unit (BPU) instruction is generated based on the load / store instruction with compression instruction and the neural network model sample (specifically, based on the model characteristics of the neural network model sample).

[0105] For example, in actual application, when the BPU needs to store, if the encoding function based on the compression instruction is specified, the compression method of the neural network model is determined based on the compression instruction and the model characteristics of the neural network model, and the code number corresponding to the compression method is loaded into the header of the data to be compressed. For example, 00 represents uncompressed data encoding, 01 represents Huffman encoding, and 10 represents zero-ary encoding. When the BPU needs to load, first determine the code number in the compressed data to be loaded, and then generate the corresponding decoding instruction.

[0106] Exemplarily, the Huffman compression coding algorithm and the zero-ary compression coding algorithm mentioned in the above embodiments are respectively loaded into specific hardware modules integrated in the BPU as additional functions of load / store, thereby further accelerating the execution speed of encoding and / or decoding.

[0107] Fig.12 FIG. 1 is a flow chart of a data processing method provided by an exemplary embodiment of the present disclosure. Fig.12 As shown, the data processing method provided by the embodiment of the present disclosure includes the following steps.

[0108] Step S400, determining the model characteristics of the neural network model to be processed.

[0109] It can be understood that the neural network model to be processed mentioned in step S400 refers to a model that requires compression processing of the data to be migrated output by the model based on compression instructions.

[0110] Exemplarily, the data to be migrated includes image data obtained by processing the image to be processed using a neural network model (which can be regarded as the neural network model to be processed), and the neural network model includes an image recognition model and / or an image processing model. Alternatively, the data to be migrated includes audio data obtained by processing the audio to be processed using a neural network model (which can be regarded as the neural network model to be processed), and the neural network model includes an audio processing model.

[0111] Step S500, based on the model features of the neural network model to be processed and the compression instruction, a compression operation is performed on the data to be migrated output by the neural network model to be processed to obtain compressed data.

[0112] Exemplarily, the compression instruction mentioned in step S500 is generated based on the instruction generation method mentioned in the above embodiment.

[0113] Exemplarily, in actual application, the model characteristics of the neural network model to be processed are first determined, and then based on the model characteristics of the neural network model to be processed and compression instructions, a compression operation is performed on the data to be migrated output by the network model to be processed to obtain compressed data.

[0114] The disclosed embodiments can achieve the purpose of automatically selecting a matching compression method for the data to be migrated output by the neural network model according to the actual characteristics of the neural network model, thereby achieving the purpose of reducing the load and delay of data transmission on the bus, improving chip access efficiency, and improving the execution efficiency of the neural network model.

[0115] Fig.13 FIG. 1 is a flow chart of a data processing method provided by another exemplary embodiment of the present disclosure. Fig.12 The present disclosure is extended based on the embodiment shown Fig.13 The embodiment shown is described below in detail. Fig.13 The embodiment shown and Fig.12 The differences and similarities of the illustrated embodiments are not described in detail.

[0116] like Fig.13 As shown, in the embodiment of the present disclosure, after the compressed data is obtained, the following steps are also included.

[0117] Step S600, determining a compression method corresponding to the compressed data.

[0118] For example, the compression method corresponding to the compressed data is determined based on the compression method information in the compressed data (also called the data packet to be migrated).

[0119] Step S700: decompressing the compressed data based on the compression method corresponding to the compressed data to obtain the data to be migrated.

[0120] Step S800: Processing the data to be migrated.

[0121] Since the compressed data is generated based on the compression instruction mentioned in the above embodiment, the embodiment of the present disclosure can greatly improve the decompression speed.

[0122] Exemplary Devices

[0123] Combination of the above Figures 3 to 13 , describes the method embodiment of the present application in detail, and the following is combined with Fig.14 and Fig.15 , describes the device embodiment of the present application in detail. It should be understood that the description of the method embodiment corresponds to the description of the device embodiment, so the parts not described in detail can refer to the previous method embodiment.

[0124] Fig.14 FIG. 1 is a schematic diagram of the structure of an instruction generation device provided by an exemplary embodiment of the present disclosure. Fig.14 As shown, the instruction generation device provided by the embodiment of the present disclosure includes a model feature determination module 100, a compression method determination module 200 and a compression instruction generation module 300.

[0125] Specifically, the model feature determination module 100 is used to determine the model features of at least one neural network model sample. The compression method determination module 200 is used to determine the compression method of at least one neural network model sample based on the model features of at least one neural network model sample. The compression instruction generation module 300 is used to generate a compression instruction based on the model features and compression method of at least one neural network model sample.

[0126] In some embodiments, the model feature determination module 100 is also used to determine the hierarchical structure of at least one neural network model sample, and determine the model features of at least one neural network model sample based on the hierarchical structure of at least one neural network model sample.

[0127] In some embodiments, the model feature determination module 100 is also used to determine, for each neural network model sample in at least one neural network model sample, whether the neural network model sample includes a convolution layer based on the hierarchical structure of the neural network model sample, and if it is determined that the neural network model sample includes a convolution layer, the model feature of the neural network model sample is characterized by the parameter standard deviation of the convolution kernel in the convolution layer. In addition, the compression method determination module 200 is also used to calculate the parameter standard deviation of the convolution kernel in the convolution layer, determine whether the parameter standard deviation meets the preset conditions, and if the parameter standard deviation meets the preset conditions, determine that the compression method corresponding to the neural network model sample is the Huffman coding method.

[0128] In some embodiments, the compression method determination module 200 is also used to determine the number of bytes of the data sample to be migrated output by the neural network model sample, and judge whether the number of bytes is greater than a preset byte number threshold. If the parameter standard deviation meets the preset conditions and the number of bytes is greater than the preset byte number threshold, it is determined that the compression method corresponding to the neural network model sample is the Huffman coding method.

[0129] In some embodiments, the model feature determination module 100 is also used to determine, for each neural network model sample in at least one neural network model sample, whether the neural network model sample includes an activation function layer based on the hierarchical structure of the neural network model sample, and if it is determined that the neural network model sample includes an activation function layer, the activation function included in the activation function layer is used to characterize the model features of the neural network model sample. In addition, the compression mode determination module 200 is also used to determine, based on the activation function, that the compression mode corresponding to the neural network model sample is a zero-ary compression encoding mode.

[0130] Fig.15 FIG. 2 is a schematic diagram of the structure of a data processing device provided by an exemplary embodiment of the present disclosure. Fig.15 As shown, the data processing device provided by the embodiment of the present disclosure includes a determination module 400 and a compression module 500 .

[0131] Specifically, the determination module 400 is used to determine the model features of the neural network model to be processed. The compression module 500 is used to compress the data to be migrated output by the neural network model to be processed based on the model features of the neural network model to be processed and the compression instruction to obtain compressed data.

[0132] In some embodiments, the data processing device further includes a processing module, the processing module being used to determine a compression method corresponding to the compressed data, decompress the compressed data based on the compression method corresponding to the compressed data, obtain the data to be migrated, and process the data to be migrated.

[0133] Exemplary Electronic Devices

[0134] Below, reference Fig.16 An electronic device according to an embodiment of the present disclosure is described. Fig.16 A block diagram of an electronic device according to an embodiment of the present disclosure is illustrated.

[0135] like Fig.16 As shown, the electronic device 90 includes one or more processors 901 and a memory 902 .

[0136] The processor 901 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 90 to perform desired functions.

[0137] The memory 902 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 901 may run the program instructions to implement the instruction generation method and / or data processing method of the various embodiments of the present disclosure described above and / or other desired functions. Various contents such as data to be migrated may also be stored in the computer-readable storage medium.

[0138] In one example, the electronic device 90 may further include: an input device 903 and an output device 904 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0139] The input device 903 may include, for example, a keyboard, a mouse, etc.

[0140] The output device 904 can output various information to the outside, including prediction information, etc. The output device 904 can include, for example, a display, a communication network and a remote output device connected thereto, etc.

[0141] Of course, to simplify, Fig.16 Only some of the components related to the present disclosure in the electronic device 90 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 90 may also include any other appropriate components.

[0142] Exemplary computer program products and computer-readable storage media

[0143] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the instruction generation method and / or data processing method according to various embodiments of the present disclosure described in the above "Exemplary Method" section of this specification.

[0144] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0145] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, causes the processor to execute the steps in the instruction generation method and / or data processing according to various embodiments of the present disclosure described in the above “Exemplary Method” section of this specification.

[0146] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0147] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0148] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0149] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0150] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0151] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for generating an instruction, comprising: Determine a model feature of each of at least one neural network model sample, wherein the model feature includes a parameter standard deviation of a convolution kernel in a convolution layer or an activation function included in an activation function layer; Determining a compression method for each of the at least one neural network model samples based on the model characteristics of each of the at least one neural network model samples; Based on the model characteristics and compression method of each of the at least one neural network model samples, a compression instruction is generated, wherein the compression instruction is used to compress the data to be migrated output by the neural network model based on the model characteristics of the neural network model; The determining, based on the model features of each of the at least one neural network model samples, a compression method for each of the at least one neural network model samples, comprises: If it is determined based on the hierarchical structure of the neural network model sample that the neural network model sample includes a convolution layer, then the parameter standard deviation of the convolution kernel in the convolution layer is calculated; if the parameter standard deviation meets the preset conditions, it is determined that the compression method corresponding to the neural network model sample is the Huffman coding method; or, If it is determined based on the hierarchical structure of the neural network model sample that the neural network model sample includes an activation function layer, then it is determined based on the activation function that the compression method corresponding to the neural network model sample is a zero-ary compression encoding method.

2. The method according to claim 1, wherein: The determining of the model characteristics of at least one neural network model sample comprises: Determining a hierarchical structure of each of the at least one neural network model samples; Based on the hierarchical structure of each of the at least one neural network model samples, the model characteristics of each of the at least one neural network model samples are determined.

3. The method according to claim 2, wherein: The determining of the model features of each of the at least one neural network model samples based on the hierarchical structure of each of the at least one neural network model samples comprises: For each neural network model sample in the at least one neural network model sample, If it is determined based on the hierarchical structure of the neural network model sample that the neural network model sample includes a convolution layer, the parameter standard deviation of the convolution kernel in the convolution layer is used to characterize the model characteristics of the neural network model sample.

4. The method according to claim 2, wherein: The determining of the model features of each of the at least one neural network model samples based on the hierarchical structure of each of the at least one neural network model samples comprises: For each neural network model sample in the at least one neural network model sample, If it is determined based on the hierarchical structure of the neural network model sample that the neural network model sample includes an activation function layer, the activation function included in the activation function layer is used to characterize the model characteristics of the neural network model sample.

5. The method according to any one of claims 1 to 4, wherein: The compression instruction includes: an instruction to store compression method information that can characterize the compression method corresponding to the neural network model sample into the compressed and generated data packet to be migrated.

6. A data processing method, comprising: Determining model characteristics of the neural network model to be processed; Based on the model characteristics and compression instructions of the neural network model to be processed, a compression operation is performed on the data to be migrated output by the network model to be processed to obtain compressed data, wherein the compression instruction is generated based on the instruction generation method described in any one of claims 1 to 5 above.

7. The method according to claim 6, wherein: After obtaining the compressed data, the method further includes: Determining a compression method corresponding to the compressed data; decompressing the compressed data based on a compression method corresponding to the compressed data to obtain data to be migrated; The data to be migrated is processed.

8. The method according to claim 6, wherein: The data to be migrated includes image data obtained after the image to be processed is processed by using the neural network model, and the neural network model includes an image recognition model and / or an image processing model; and / or, The data to be migrated includes audio data obtained after the processed audio is processed using the neural network model, and the neural network model includes an audio processing model.

9. An instruction generating device, comprising: A model feature determination module, used to determine the model features of at least one neural network model sample, wherein the model features include the parameter standard deviation of the convolution kernel in the convolution layer or the activation function included in the activation function layer; A compression mode determination module, used to determine the compression mode of each of the at least one neural network model samples based on the model characteristics of each of the at least one neural network model samples; A compression instruction generation module, used to generate a compression instruction based on the model characteristics and compression method of each of the at least one neural network model samples, wherein the compression instruction is used to perform a compression operation on the data to be migrated output by the neural network model based on the model characteristics of the neural network model; The determining, based on the model features of each of the at least one neural network model samples, a compression method for each of the at least one neural network model samples, comprises: If it is determined based on the hierarchical structure of the neural network model sample that the neural network model sample includes a convolution layer, then the parameter standard deviation of the convolution kernel in the convolution layer is calculated; if the parameter standard deviation meets the preset conditions, it is determined that the compression method corresponding to the neural network model sample is the Huffman coding method; or, If it is determined based on the hierarchical structure of the neural network model sample that the neural network model sample includes an activation function layer, then it is determined based on the activation function that the compression method corresponding to the neural network model sample is a zero-ary compression encoding method.

10. A data processing device, comprising: A determination module, used for determining the model characteristics of the neural network model to be processed; A compression module is used to compress the data to be migrated output by the neural network model to be processed based on the model characteristics and compression instructions of the neural network model to be processed to obtain compressed data, wherein the compression instructions are generated based on the instruction generation method described in any one of claims 1 to 5 above.

11. A computer-readable storage medium storing a computer program, wherein the computer program is used to execute the method according to any one of claims 1 to 8.

12. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to execute the method according to any one of claims 1 to 8.

Images