Model indexing method and device, electronic equipment and computer readable medium

Abstract

The embodiment of the invention discloses a model indexing method and device, electronic equipment and a computer readable medium. A specific embodiment of the method comprises the steps of receiving target neural network model information; generating each initial mass center number based on the target neural network model information; based on the target neural network model information and the number of each initial centroid, generating each centroid information group and compression standard model information; generating an index code matching block set based on the target neural network model information and each centroid information block; sending the index pair code set to a display terminal, so that the display terminal displays the received index pair code set; storing the index pair code set and the compression standard model information to a data storage server; performing data conversion processing on the index code block set to obtain an index lookup table; and storing the index lookup table to a hardware storage structure. According to the embodiment, the calculation speed of the compression model is increased, and calculation resources needed by the model are reduced.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a model indexing method, device, electronic device, and computer-readable medium. Background Art

[0002] Deep Neural Networks (DNN) have achieved remarkable success in fields such as computer vision and natural language processing. However, this performance improvement comes at the expense of extremely energy-intensive DNN models as the model size becomes larger and larger. For example, state-of-the-art DNNs such as GPT-3 may require GB-level model size and 102GFLOPs of inference computation. Therefore, the deployment of these DNNs remains a challenge and requires the support of key technologies. Model compression technology is one of the key technologies, which achieves a better trade-off between accuracy and hardware overhead through algorithm and hardware co-design. Currently, quantization methods are often used for model compression, which converts floating-point weights and activations in the model into low-precision representations (such as 8-bit integers) to reduce model size and computational requirements.

[0003] However, when using the above-mentioned deep neural network to compress the model, the following technical problems often occur:

[0004] First, the way the original model is tested and used after compression is to calculate through the low-precision weights and floating-point numbers in the compressed model, so that the actual compression calculation method of the model after compression is consistent with the original model, resulting in a slower calculation speed of the compressed model.

[0005] Second, converting the floating-point weights and activations in the model into low-precision representations requires a larger bit width, so that the actual compressed size of the model after compression is still large, resulting in greater computing resources required to use the compressed model.

[0006] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0007] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0008] Some embodiments of the present disclosure propose a model indexing method, device, electronic device and computer-readable medium for deep neural network model compression to solve one or more of the technical problems mentioned in the above background technology section.

[0009] In a first aspect, some embodiments of the present disclosure provide a model indexing method for deep neural network model compression, the method comprising: receiving target neural network model information, wherein the target neural network model information comprises information of each neural network layer, each neural network layer information in the each neural network layer information comprises activation layer information and weight layer information, the activation layer information comprises each activation information, and the weight layer information comprises each weight information; based on the target neural network model information, generating each initial number of centroids; based on the target neural network model information and each initial number of centroids, generating each centroid information group and compressed standard model information; based on the target neural network model information and each centroid information group, generating an index pair group set; sending the index pair group set to a display terminal, so that the display terminal displays the received index pair group set; storing the index pair group set and the compressed standard model information to a data storage server; performing data conversion processing on the index pair group set to obtain an index lookup table; storing the index lookup table to a hardware storage structure.

[0010] In a second aspect, some embodiments of the present disclosure provide a model indexing device for deep neural network model compression, the device comprising: a receiving unit, configured to receive target neural network model information, wherein the target neural network model information comprises information of various neural network layers, each of the neural network layer information comprises activation layer information and weight layer information, the activation layer information comprises various activation information, and the weight layer information comprises various weight information; a first generating unit, configured to generate various initial centroid numbers based on the target neural network model information; a second generating unit, configured to generate various initial centroid numbers based on the target neural network model information and the various initial centroids; The number of centroids is determined by the number of centroids, and each centroid information group and compressed standard model information are generated; the third generating unit is configured to generate an index pair group set based on the target neural network model information and the each centroid information group; the sending unit is configured to send the index pair group set to the display terminal, so that the display terminal displays the received index pair group set; the first storage unit is configured to store the index pair group set and the compressed standard model information to the data storage server; the data conversion processing unit is configured to perform data conversion processing on the index pair group set to obtain an index lookup table; the second storage unit is configured to store the index lookup table to the hardware storage structure.

[0011] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors, wherein each of the one or more processors comprises one or more processing units, an activation buffer, a weight buffer, an accumulator, one or more encoders and an output buffer; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the first aspect.

[0012] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0013] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through a model indexing method for deep neural network model compression in some embodiments of the present disclosure, the calculation speed of the compressed model is improved. Specifically, the reason for the slow calculation speed of the compressed model is that the way the original model is tested and used after compression is calculated by low-precision weights and floating-point numbers in the compressed model, so that the actual compression calculation method after the model is compressed is consistent with the original model, resulting in a slow calculation speed of the compressed model. Based on this, a model indexing method for deep neural network model compression in some embodiments of the present disclosure, first, receives target neural network model information, wherein the above-mentioned target neural network model information includes each neural network layer information, each neural network layer information in the above-mentioned each neural network layer information includes activation layer information and weight layer information, the above-mentioned activation layer information includes each activation information, and the above-mentioned weight layer information includes each weight information. Thus, the model data information that needs to be compressed can be obtained. Secondly, based on the above-mentioned target neural network model information, each initial number of centroids is generated. Thus, the different initial numbers of centroids corresponding to each neural network layer can be obtained. Then, based on the target neural network model information and the initial number of centroids, generate each centroid information group and compressed standard model information. Thus, the centroid information group corresponding to each neural network layer and the compressed model information after compression can be obtained. Then, based on the target neural network model information and the centroid information groups, generate an index pair group set. Thus, the index pair group set corresponding to the target neural network model can be obtained. Secondly, send the index pair group set to the display terminal so that the display terminal displays the received index pair group set. Thus, the index pair group set can be displayed. Then, store the index pair group set and the compressed standard model information in a data storage server. Thus, the generated index pair group set and compressed standard model information can be stored. Then, perform data conversion processing on the index pair group set to obtain an index lookup table. Thus, the index pair group set can be converted into an index lookup table. Secondly, store the index lookup table in a hardware storage structure. Thus, the obtained index lookup table can be stored in a hardware storage structure. Also, because the weights and activations of each layer in the model are recompressed by building an index pair set, the model calculation is actually performed through the index lookup table when the compressed model is actually used for calculation, thereby improving the calculation speed of the compressed model. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0015] Figure 1 A flowchart of some embodiments of a model index for deep neural network model compression according to the present disclosure is shown; Figure 2 It is a structural schematic diagram of some embodiments of a model indexing device for deep neural network model compression according to the present disclosure; Figure 3 is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure; Figure 4 It is a structural schematic diagram of a processing device included in an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0019] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0020] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0021] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0022] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0023] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0024] Figure 1 A process 100 of some embodiments of a model indexing method for deep neural network model compression according to the present disclosure is shown. The model indexing method for deep neural network model compression includes the following steps:

[0025] Step 101, receiving target neural network model information.

[0026] In some embodiments, the execution subject (e.g., a computing device) of the model indexing method for deep neural network model compression may receive target neural network model information, wherein the execution subject may receive the target neural network model information sent by a data storage server. The data storage server may be a server for managing and storing data. The target neural network model information may be neural network model information. For example, the neural network model information may be, but is not limited to, information such as the network structure and activation weight parameters of a DNN model or a Transformer model. For example, the DNN model may be VGG16 or ResNet18 / 50. The Transformer model may be ViT or Bert. The target neural network model information may include information of each neural network layer. Each neural network layer information in the above-mentioned individual neural network layer information may include activation layer information and weight layer information. The above-mentioned activation layer information may be used to characterize the activation layer connection mode, activation layer data information, and activation layer function information. The above-mentioned weight layer information may be used to characterize the weight layer connection mode, weight layer data information, and weight layer function information. The above-mentioned activation layer connection mode may be fully connected. The above-mentioned activation layer information may include individual activation information, activation layer connection mode, and activation layer function information. The above-mentioned individual activation information may characterize the above-mentioned activation layer data information. The activation layer function information may be a Sigmoid function or a ReLU function. The activation information may be a floating point value used to characterize the activation. For example, the activation information may be a 32-bit floating point value. The weight layer connection mode may be a full connection. The weight layer information may include each weight information, the weight layer connection mode and the weight layer function information. Each weight information may characterize the weight layer data information. The weight layer function information may be a Sigmoid function or a ReLU function. The weight information may be a floating point value used to characterize the weight. For example, the weight information may be a 32-bit floating point value.

[0027] Step 102, generating each initial number of centroids based on the target neural network model information.

[0028] In some embodiments, the execution subject may generate each initial number of centroids based on the target neural network model information. Among the initial numbers of centroids, one of the initial numbers of centroids may correspond to a neural network layer information. The initial number of centroids may be the number of centroids required for the activation layer and the weight layer corresponding to the corresponding neural network layer information when performing clustering processing.

[0029] In some optional implementations of some embodiments, the execution subject may generate each initial number of centroids based on the target neural network model information through the following steps:

[0030] The first step is to perform the following steps for each neural network layer information in the above target neural network model information:

[0031] The first sub-step is to generate a sample distribution range based on the above-mentioned neural network layer information. Among them, the above-mentioned sample distribution range may include an activation sample distribution range and a weight sample distribution range. In practice, first, the above-mentioned execution entity may obtain the activation layer information and weight layer information included in the above-mentioned neural network layer information. Secondly, the above-mentioned execution entity may obtain the numerical range of the activation information included in the above-mentioned activation layer information. Then, the above-mentioned execution entity may determine the numerical range of the obtained activation information as the activation sample distribution range. Then, the above-mentioned execution entity may obtain the numerical range of the weight information included in the above-mentioned weight layer information. Secondly, the above-mentioned execution entity may determine the numerical range of the obtained weight information as the weight sample distribution range. Finally, the above-mentioned execution entity may combine the above-mentioned activation sample distribution range and the above-mentioned weight sample distribution range into a sample distribution range.

[0032] The second sub-step is to generate the initial number of activated centroids based on the sample distribution range and the activation layer information included in the neural network layer information, wherein the initial number of activated centroids may be the number of centroids corresponding to the activation layer information used for clustering processing.

[0033] The third sub-step is to generate the initial number of weighted centroids based on the sample distribution range and the weight layer included in the neural network layer information. The initial number of weighted centroids may be the number of centroids corresponding to the weight layer information used for clustering, and the number of centroids may be an integer. The method for generating the initial number of weighted centroids may refer to the method for generating the initial number of activated centroids.

[0034] The fourth sub-step is to integrate the initial number of activated centroids and the initial number of weighted centroids to obtain an initial number of centroids. In practice, the execution entity may combine the initial number of activated centroids and the initial number of weighted centroids to obtain an initial number of centroids.

[0035] In some optional implementations of some embodiments, the execution subject may generate the initial number of activation centroids based on the sample distribution range and the activation layer information included in the neural network layer information through the following steps:

[0036] The first step is to generate the first expected clustering information and each first expected distance information based on the first preset number of centroids, the above-mentioned sample distribution range and the preset number of samplings. Among them, the above-mentioned first preset number of centroids can be numerical data. The above-mentioned first preset number of centroids can be preset, which is not limited here. For example, the above-mentioned first preset number of centroids can be "1". The above-mentioned preset number of samplings can be numerical data. The above-mentioned preset number of samplings can be preset, which is not limited here. For example, the above-mentioned preset number of samplings can be "20". The above-mentioned first expected clustering information can be an expectation for characterizing the above-mentioned each first expected distance information. The above-mentioned first expected clustering information can be numerical data. Each of the first expected distance information in the above-mentioned each first expected distance information can be numerical data.

[0037] The second step is to generate the first centroid clustering information based on the first preset number of centroids and the activation layer information included in the above neural network layer information. Among them, the above first centroid clustering information can be numerical data. In practice, first, the above execution subject can cluster the activation layer information included in the above neural network layer information through a clustering algorithm to obtain each activation cluster cluster and each activation cluster cluster center. Among them, each activation cluster cluster in the above activation cluster clusters has a one-to-one correspondence with each activation cluster cluster center in the above activation cluster cluster centers. For example, the above clustering algorithm can be a K-means algorithm. Then, the above execution subject can determine the absolute value of the difference between the sample in each activation cluster cluster and the corresponding activation cluster cluster center as the sample distance. Then, the above execution subject can determine the sum of the determined sample distances as the total sample distance. Then, the above execution subject can determine the logarithm of the total sample distance as the first centroid clustering information.

[0038] The third step is to generate first clustering information based on the first expected clustering information and the first centroid clustering information. In practice, the execution subject may input the first expected clustering information and the first centroid clustering information into a first preset formula to obtain the first clustering information. The first preset formula may be Among them, the first clustering information can be expressed as Gap k Gap k It may be the first clustering information when the first preset number of centroids is k. It can be the first expected distance information. It may be the first expected clustering information corresponding to each first expected distance information. kIt can be calculated as the sum of the total distances of each sample in the activated cluster to the corresponding activated cluster center. log(D k ) can be the first centroid clustering information.

[0039] The fourth step is to generate a first standard deviation based on the first expected clustering information, the first expected distance information and the preset sampling times. In practice, the execution subject may input the first expected clustering information, the first expected distance information and the preset sampling times into a second preset formula to obtain the first standard deviation. The second preset formula may be s k It can be the first standard deviation when the first preset number of centroids is k. B can be the preset number of sampling times. The bth first expected distance information

[0040] Step 5: For the activation layer information included in the above neural network layer information, perform the following centroid loop steps:

[0041] The first sub-step is to generate the second expected clustering information and each second expected distance information based on the second preset number of centroids, the above-mentioned sample distribution range and the above-mentioned preset number of sampling times. Among them, the generation method of the above-mentioned second expected clustering information and each second expected distance information can refer to the generation method of the above-mentioned first expected clustering information and each first expected distance information. The above-mentioned second preset number of centroids can be numerical data. The above-mentioned second preset number of centroids can be pre-set, which is not limited here. For example, the above-mentioned second preset number of centroids can be "2". The above-mentioned second expected clustering information can be an expectation for characterizing the above-mentioned each second expected distance information. The above-mentioned second expected clustering information can be numerical data. Each second expected distance information in the above-mentioned each second expected distance information can be numerical data.

[0042] The second sub-step is to generate second centroid clustering information based on the second preset number of centroids and the activation layer information included in the neural network layer information. The second centroid clustering information may be generated in a manner similar to the first centroid clustering information.

[0043] The third sub-step is to generate second clustering information based on the second expected clustering information and the second centroid clustering information. The second clustering information may be generated in the same manner as the first clustering information.

[0044] The fourth sub-step is to generate a second standard deviation based on the second expected clustering information, the second expected distance information and the preset sampling times. The second standard deviation may be generated by referring to the first standard deviation.

[0045] The fifth sub-step, in response to determining that the first clustering information, the second clustering information and the second standard deviation do not meet the preset difference condition, determines the second clustering information as the first clustering information to update the first clustering information, determines the second preset number of centroids as the first preset number of centroids to update the first preset number of centroids, and adjusts the second preset number of centroids, and uses the adjusted second preset number of centroids to perform the centroid loop step again. The preset difference condition may be that the first clustering information is greater than or equal to the difference between the second clustering information and the second standard deviation. The method of adjusting the second preset number of centroids may be to add one to the second preset number of centroids.

[0046] The sixth sub-step is, in response to determining that the first clustering information, the second clustering information and the second standard deviation satisfy the preset difference condition, determining the first preset number of centroids as the initial activation number of centroids.

[0047] In some optional implementations of some embodiments, the execution subject may generate the first expected clustering information and each first expected distance information based on the first preset number of centroids, the sample distribution range, and the preset number of sampling times through the following steps:

[0048] The first step is to perform the following steps based on the above preset sampling times:

[0049] The first sub-step is to generate each random sample information based on the sample distribution range. The random sample information may be numerical data. In practice, the execution subject may generate random sample information corresponding to the activation information number uniformly based on the sample distribution range and the activation information number of the activation layer information included in the neural network layer information.

[0050] The second sub-step is to generate each sample cluster based on the above-mentioned random sample information and the above-mentioned first preset number of centroids. Among them, the above-mentioned each sample cluster can be a cluster obtained after clustering processing. Each sample cluster in the above-mentioned each sample cluster includes a corresponding sample cluster center. In practice, the above-mentioned execution body can perform clustering processing on the above-mentioned each random sample information through a clustering algorithm according to the above-mentioned first preset number of centroids to obtain each sample cluster. For example, the above-mentioned clustering algorithm can be a K-means algorithm.

[0051] The third sub-step is to generate each sample distance information based on the above-mentioned each random sample information and the above-mentioned each sample cluster. The above-mentioned each sample distance information can be the distance from each random sample information in each random sample information to the center of the sample cluster corresponding to the above-mentioned random sample information. In practice, the absolute value of the difference between each random sample information in the above-mentioned each random sample information and the center of the cluster included in the corresponding sample cluster is determined as the sample distance information.

[0052] The fourth sub-step is to determine the sum of the above-mentioned distance information of each sample as the first expected distance information.

[0053] The second step is to generate first expected clustering information based on the determined first expected distance information and the preset sampling times. In practice, the execution subject may input the first expected distance information and the preset sampling times into a third preset formula to obtain the first expected clustering information. The third preset formula may be It is the first expected clustering information corresponding to each first expected distance information.

[0054] Step 103, based on the target neural network model information and the initial number of centroids, generate each centroid information group and compressed standard model information.

[0055] In some embodiments, the execution entity may generate various centroid information groups and compressed standard model information based on the target neural network model information and the initial number of centroids. Each of the various centroid information groups may be a centroid information group obtained after the activation layer information and weight layer information included in each neural network layer are processed by a clustering algorithm. Each centroid information in the centroid information group may be a numerical value. The clustering algorithm may be a K-means clustering algorithm. The compressed standard model information may be a compressed model and compressed model information obtained after the target neural network model information is compressed.

[0056] In some optional implementations of some embodiments, the execution subject may generate each centroid information group and compressed standard model information based on the target neural network model information and each initial centroid number through the following steps:

[0057] The first step is to determine the above target neural network model information as the initial neural network model information.

[0058] The second step is to input the preset calibration data set into the initial neural network model corresponding to the above-mentioned initial neural network model information to obtain an initial accuracy rate. Among them, the above-mentioned preset calibration data set can be a data set used to calibrate and test the neural network model. The specific content of the above-mentioned preset calibration data set can be pre-set and is not limited here. The above-mentioned initial accuracy rate can be the accuracy rate of the model obtained by characterizing the preset calibration data set in the calculation of the initial neural network model. In practice, the above-mentioned execution entity can input the preset calibration data set into the above-mentioned initial neural network model to obtain the initial accuracy rate corresponding to the above-mentioned initial neural network model.

[0059] In the third step, for the above initial neural network model information, perform the following compression model loop:

[0060] In the first sub-step, for each neural network layer information in the above initial neural network model information, the following model layer compression steps are performed:

[0061] Sub-step 1, based on the neural network layer information and the number of initial centroids corresponding to the neural network layer information, generate compression model layer information, each initial activation centroid information and each initial weight centroid information corresponding to the neural network layer information. Among them, each initial activation centroid information in the initial activation centroid information can be the centroid information obtained by using a clustering algorithm for the activation layer information included in the neural network layer information. Each initial weight centroid information in the initial weight centroid information can be the centroid information obtained by using a clustering algorithm for the weight layer information included in the neural network layer information. The clustering algorithm can be a K-means clustering algorithm. The compression model layer information can be the neural network layer information obtained after compression processing of the neural network layer information. In practice, first, the execution subject can cluster the activation layer information included in the neural network layer information through a clustering algorithm to obtain each initial activation centroid information. Then, the execution subject can cluster the weight layer information included in the neural network layer information through a clustering algorithm to obtain each initial weight centroid information. For example, the clustering algorithm can be a K-means algorithm. Then, the execution entity may replace the activation information and the initial activation centroid information included in the activation layer information to obtain the replaced activation layer information. The replacement process may be to extract the initial activation centroid information closest to the activation information from the initial activation centroid information for each of the activation information, so as to update the activation information. The method of replacing the weight layer information may refer to the method of replacing the activation layer information. Secondly, the execution entity may combine the replaced activation layer information and the replaced weight layer information into compressed model layer information.

[0062] Sub-step 2: Integrate the above-mentioned initial activation centroid information and the above-mentioned initial weight centroid information to obtain an initial centroid information group. In practice, the above-mentioned execution subject can combine the above-mentioned initial activation centroid information and the above-mentioned initial weight centroid information to obtain an initial centroid information group.

[0063] The second sub-step is to integrate the obtained compression model layer information to obtain the compression model and compression model information. In practice, first, the execution subject can determine the neural network model obtained by combining the neural network layers corresponding to the above compression model layer information as the compression model. Then, the execution subject can combine the above compression model layer information into the compression model information.

[0064] The third sub-step is to input the preset calibration data set into the compression model to obtain a compression accuracy rate, wherein the compression accuracy rate may be an accuracy rate of a model obtained by calculating the preset calibration data set in the compression model.

[0065] The fourth sub-step, in response to determining that the above-mentioned initial accuracy rate and the above-mentioned compression accuracy rate do not meet the preset accuracy loss condition, adjust the number of each initial centroid, and use the adjusted number of each initial centroid to execute the above-mentioned compression model cycle again. Among them, the above-mentioned preset accuracy loss condition can be that the difference between the initial accuracy rate and the compression accuracy rate meets the preset accuracy range. For example, the above-mentioned preset accuracy range can be "0.4%-0.5%". In practice, the above-mentioned execution subject can use the gradient descent method (such as a small batch gradient descent algorithm) to adjust the above-mentioned number of each initial centroid. The above-mentioned execution subject can adjust the above-mentioned number of each initial centroid according to the position of the difference in the above-mentioned preset accuracy range. For example, when the above-mentioned difference is 0.3%, the above-mentioned execution subject can reduce the number of initial centroids. When the above-mentioned difference is 0.6%, the above-mentioned execution subject can increase the number of initial centroids.

[0066] In a fifth sub-step, in response to determining that the initial accuracy and the compression accuracy meet the preset accuracy loss condition, the generated initial centroid information groups are determined as centroid information groups, and the compression model and the compression model information are integrated to obtain the compression standard model information. In practice, the execution subject may combine the compression model and the compression model information to obtain the compression standard model information.

[0067] The above-mentioned generation of each centroid information group and compressed standard model information and related content as an inventive point of an embodiment of the present disclosure solves the second technical problem mentioned in the background technology: "Converting the floating-point weights and activations in the model into low-precision representations requires a larger bit width, so that the actual compressed size of the model after compression is still large, resulting in larger computing resources required to use the compressed model." The factors that lead to larger computing resources required to use the compressed model are often as follows: the weights and activations used in the model are floating-point numbers, and the computing resources required for a large number of floating-point numbers during the calculation process are large. If the above factors are solved, the effect of reducing computing resources can be achieved. In order to achieve this effect, the present disclosure determines the above-mentioned target neural network model information as the initial neural network model information. Then, the preset calibration data set is input into the initial neural network model corresponding to the above-mentioned initial neural network model information to obtain the initial accuracy. Thus, the model accuracy corresponding to the initial neural network model can be obtained. Then, for the above-mentioned initial neural network model information, the following compression model loop is performed: for each neural network layer information in the above-mentioned initial neural network model information, the following compression model layer step is performed: based on the above-mentioned neural network layer information and the initial number of centroids corresponding to the above-mentioned neural network layer information, the compression model layer information, each initial activation centroid information and each initial weight centroid information corresponding to the above-mentioned neural network layer information are generated. Thus, the relevant information corresponding to the above-mentioned neural network layer information can be obtained. Secondly, the above-mentioned initial activation centroid information and the above-mentioned initial weight centroid information are integrated to obtain an initial centroid information group. Then, the obtained compression model layer information is integrated to obtain a compression model and compression model information. Thus, the compression model and the model information corresponding to the compression model can be obtained. Then, the above-mentioned preset calibration data set is input into the above-mentioned compression model to obtain the compression accuracy. Thus, the model accuracy corresponding to the compression model can be obtained. Secondly, in response to determining that the above-mentioned initial accuracy and the above-mentioned compression accuracy do not meet the preset accuracy loss condition, the number of each initial centroid is adjusted, and the above-mentioned compression model loop is performed again using the adjusted number of each initial centroid. Thus, the number of each initial centroid can be adjusted and the compression model cycle can be executed again to generate new compression model information. Then, in response to determining that the initial accuracy and the compression accuracy meet the preset accuracy loss condition, the generated initial centroid information groups are determined as centroid information groups, and the compression model and the compression model information are integrated to obtain the compression standard model information. Also, because each neural network layer information is processed separately through the clustering algorithm, the model can be compressed with little impact on the model accuracy.Also, because the floating-point values ​​included in the original neural network layer information are clustered, more data can be represented by using a smaller bit width, reducing a large number of floating-point values, thereby reducing the computing resources required for the compression model.

[0068] Step 104, generating an index pair group set based on the target neural network model information and each centroid information group.

[0069] In some embodiments, the execution subject may generate an index pair group set based on the target neural network model information and each centroid information group, wherein the index pair group set may be a dictionary for characterizing the relationship between activation layer information and weight layer information included in each neural network layer information.

[0070] In some optional implementations of some embodiments, the execution subject may generate an index pair group set based on the target neural network model information and the centroid information groups through the following steps:

[0071] The first step is to perform the following steps for each neural network layer information in the above target neural network model information:

[0072] The first sub-step is to determine the centroid information group corresponding to the neural network layer information as the centroid information group to be indexed, wherein the centroid information group to be indexed may include each initial activation centroid information and each initial weight centroid information.

[0073] In the second sub-step, the above-mentioned to-be-indexed centroid information group is sorted and encoded to obtain an index centroid information group. The above-mentioned index centroid information group may be a dictionary for characterizing the size order of each initial activation centroid information and each initial weight centroid information. The above-mentioned index centroid information group includes an index activation centroid information group and an index weight centroid information group. In practice, the above-mentioned execution subject may sort the various initial activation centroid information included in the above-mentioned to-be-indexed centroid information group according to a preset order. The above-mentioned preset order may be from small to large. Then, the above-mentioned execution subject may encode each initial activation information in the sorted initial activation centroid information according to the above-mentioned preset order to obtain each activation code. The above-mentioned coding may be a coding method of accumulating 1 sequentially from 0. Each initial activation information corresponds to an activation code. Then, the above-mentioned execution subject combines the obtained activation codes and each initial activation information to obtain an index activation centroid information group. The method for obtaining the above-mentioned index weight centroid information group may refer to the method for obtaining the above-mentioned index activation centroid information group. Finally, the execution entity may combine the index activation centroid information group and the index weight centroid information group to obtain an index centroid information group.

[0074] The third sub-step is to perform integer coding processing on each activation information in the activation layer information included in the above neural network layer information to obtain each integer activation information. Among them, each integer activation information in the above integer activation information can be an integer value after the activation information is processed by integer coding. In practice, first, the above execution subject can sort the above activation information. Among them, the above preset order can be in ascending order. Then, the above execution subject can encode each activation information in the sorted activation information according to the above preset order to obtain each integer activation information. The above encoding can be an encoding method that accumulates 1 from 0 in sequence. Among them, each activation information corresponds to an integer activation information. For example, when there are 15 activation information (32-bit floating point values) and the above activation information is [-0.86, -1.35, ..., 2.12]. The above execution subject can sort [-0.86, -1.35, ..., 2.12] in ascending order to obtain [-1.35, -0.86, ..., 2.12]. Then, the execution entity numbers the sorted activation information to obtain integer activation information (4-5 digit integer value) [0, 1, ..., 14]. Among them, the integer activation information "0" corresponds to the activation information "-1.35", the integer activation information "1" corresponds to the activation information "-0.86", and the integer activation information "2.12" corresponds to the activation information "14".

[0075] In a fourth sub-step, each weight information in the weight layer information included in the neural network layer information is integer-encoded to obtain each integer weight information. Each integer weight information in the above-mentioned integer weight information may be an integer value after the weight information is integer-encoded. The method for obtaining the above-mentioned each integer weight information may refer to the method for obtaining the above-mentioned each integer activation information.

[0076] The fifth sub-step is to generate an index pairing group based on the above-mentioned integer activation information and the above-mentioned integer weight information. Among them, the above-mentioned index pairing group can be a dictionary that characterizes the relationship between the above-mentioned integer activation information and the above-mentioned integer weight information. In practice, first, the above-mentioned execution subject can perform the following steps for each activation information in the above-mentioned activation information: For each weight information in the above-mentioned weight information, perform the following steps: Then, the above-mentioned execution subject can determine the product of the initial activation centroid information corresponding to the above-mentioned activation information and the initial weight centroid information corresponding to the above-mentioned weight information as the centroid product information. Among them, the above-mentioned centroid product information can be a floating point value. Then, the above-mentioned execution subject can integrate the integer activation information corresponding to the above-mentioned activation information, the integer weight information corresponding to the above-mentioned weight information, and the above-mentioned centroid product information to obtain the centroid pairing information. Among them, the above-mentioned centroid pairing information can be a dictionary. In practice, the above-mentioned execution subject can combine the integer activation information corresponding to the activation information, the integer weight information corresponding to the above-mentioned weight information, and the above-mentioned centroid product information to obtain the centroid pairing information. Finally, the above-mentioned execution subject can determine the obtained centroid pairing information as an index pairing group. In practice, the above execution entity may combine the obtained centroid pairing information into an index pairing group.

[0077] In the second step, the obtained index pair groups are determined as an index pair group set.

[0078] Optionally, after step 104, the execution entity may further perform the following steps:

[0079] The first step is to generate a test accuracy rate based on the compression standard model information and the test data set in response to detecting the test data set input by the target user. The test data set may be a data set used to test the neural network model. The specific content of the test data set may be pre-set and is not limited here. The test accuracy rate may be the accuracy rate of the model obtained by calculating the compression model included in the compression standard model information for the test data set.

[0080] In the second step, in response to determining that the test accuracy meets the preset result condition, the first preset result information is sent to the display terminal, so that the display terminal displays the received first preset result information. The preset result condition may be that the test accuracy meets the preset threshold condition. For example, the preset threshold condition may be "greater than 90%." The first preset result information may be pre-set, which is not limited here. For example, the first preset result information may be "the compression model accuracy is qualified." The display terminal may be, but is not limited to, a display or a mobile phone.

[0081] In the third step, in response to determining that the test accuracy does not meet the preset result condition, the second preset result information is sent to the display terminal, so that the display terminal displays the received second preset result information. The second preset result information can be pre-set and is not limited here. For example, the second preset result information can be "the compression model accuracy is unqualified".

[0082] Step 105: Send the index pair set to the display terminal, so that the display terminal displays the received index pair set.

[0083] In some embodiments, the execution subject may send the index pair code set to a display terminal, so that the display terminal displays the received index pair code set. The display terminal may be a preset display terminal, which is not limited here. For example, the display terminal may be a display or a mobile phone.

[0084] Step 106, storing the index pair set and the compression standard model information in the data storage server.

[0085] In some embodiments, the execution entity may store the index pair set and the compression standard model information in a data storage server, wherein the data storage server may be a server for managing and storing data.

[0086] Step 107: Perform data conversion processing on the index pair set to obtain an index lookup table.

[0087] In some embodiments, the execution subject may perform data conversion processing on the index pair code set to obtain an index lookup table. The index lookup table may be a lookup table (LUT). In practice, the execution subject may convert the index pair code set into an index lookup table by using a data2mif tool.

[0088] Step 108, storing the index lookup table into the hardware storage structure.

[0089] In some embodiments, the execution entity may store the index lookup table in a hardware storage structure, wherein the hardware storage structure may be an on-chip RAM (OCRAM) included in a field programmable gate array (FPGA).

[0090] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through a model indexing method for deep neural network model compression in some embodiments of the present disclosure, the calculation speed of the compressed model is improved. Specifically, the reason for the slow calculation speed of the compressed model is that the way the original model is tested and used after compression is calculated by low-precision weights and floating-point numbers in the compressed model, so that the actual compression calculation method after the model is compressed is consistent with the original model, resulting in a slow calculation speed of the compressed model. Based on this, a model indexing method for deep neural network model compression in some embodiments of the present disclosure, first, receives target neural network model information, wherein the above-mentioned target neural network model information includes each neural network layer information, each neural network layer information in the above-mentioned each neural network layer information includes activation layer information and weight layer information, the above-mentioned activation layer information includes each activation information, and the above-mentioned weight layer information includes each weight information. Thus, the model data information that needs to be compressed can be obtained. Secondly, based on the above-mentioned target neural network model information, each initial number of centroids is generated. Thus, the different initial numbers of centroids corresponding to each neural network layer can be obtained. Then, based on the target neural network model information and the initial number of centroids, generate each centroid information group and compressed standard model information. Thus, the centroid information group corresponding to each neural network layer and the compressed model information after compression can be obtained. Then, based on the target neural network model information and the centroid information groups, generate an index pair group set. Thus, the index pair group set corresponding to the target neural network model can be obtained. Secondly, send the index pair group set to the display terminal so that the display terminal displays the received index pair group set. Thus, the index pair group set can be displayed. Then, store the index pair group set and the compressed standard model information in a data storage server. Thus, the generated index pair group set and compressed standard model information can be stored. Then, perform data conversion processing on the index pair group set to obtain an index lookup table. Thus, the index pair group set can be converted into an index lookup table. Secondly, store the index lookup table in a hardware storage structure. Thus, the obtained index lookup table can be stored in a hardware storage structure. Also, because the weights and activations of each layer in the model are recompressed by building an index pair set, the model calculation is actually performed through the index lookup table when the compressed model is actually used for calculation, thereby improving the calculation speed of the compressed model.

[0091] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a model indexing device for deep neural network model compression, and these device embodiments are similar to Figure 1Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0092] like Figure 2 As shown, some embodiments of the model indexing device 200 for deep neural network model compression include: a receiving unit 201, a first generating unit 202, a second generating unit 203, a third generating unit 204, a sending unit 205, a first storage unit 206, a data conversion processing unit 207 and a second storage unit 208. The receiving unit 201 is configured to receive target neural network model information, wherein the target neural network model information includes information of each neural network layer, each neural network layer information in the above-mentioned neural network layer information includes activation layer information and weight layer information, the activation layer information includes each activation information, and the weight layer information includes each weight information; the first generating unit 202 is configured to generate each initial number of centroids based on the target neural network model information; the second generating unit 203 is configured to generate each centroid information group and compression standard model information based on the target neural network model information and each initial number of centroids. ; The third generation unit 204 is configured to generate an index pair group set based on the above-mentioned target neural network model information and the above-mentioned each centroid information group; the sending unit 205 is configured to send the above-mentioned index pair group set to the display terminal, so that the above-mentioned display terminal displays the received above-mentioned index pair group set; the first storage unit 206 is configured to store the above-mentioned index pair group set and the above-mentioned compressed standard model information to the data storage server; the data conversion processing unit 207 is configured to perform data conversion processing on the above-mentioned index pair group set to obtain an index lookup table; the second storage unit 208 is configured to store the above-mentioned index lookup table to the hardware storage structure.

[0093] It is understood that the units described in the device 200 are similar to those described in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the device 200 and the units included therein, and will not be described in detail here.

[0094] Reference below Figure 3 , which shows a structural schematic diagram of an electronic device 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0095] like Figure 3As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0096] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0097] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0098] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0099] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0100] The computer-readable medium may be included in the electronic device; or it may exist independently without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: receives target neural network model information, wherein the target neural network model information includes information of each neural network layer, each neural network layer information in the neural network layer information includes activation layer information and weight layer information, the activation layer information includes activation information, and the weight layer information includes weight information; generates initial centroid numbers based on the target neural network model information; generates centroid information groups and compressed standard model information based on the target neural network model information and the initial centroid numbers; generates index pair code group sets based on the target neural network model information and the centroid information groups; sends the index pair code group set to the display terminal, so that the display terminal displays the received index pair code group set; stores the index pair code group set and the compressed standard model information in the data storage server; performs data conversion processing on the index pair code group set to obtain an index lookup table; stores the index lookup table in the hardware storage structure.

[0101] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0102] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0103] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor including a receiving unit, a first generating unit, a second generating unit, a third generating unit, a sending unit, a first storage unit, a data conversion processing unit, and a second storage unit. Among them, the names of these units do not constitute a limitation on the units themselves in certain cases. For example, the receiving unit may also be described as a "unit for receiving target neural network model information."

[0104] Further references Figure 4 It shows a schematic structural diagram of a processing device 301 included in an electronic device 300 suitable for implementing some embodiments of the present disclosure. Figure 4 The processing device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0105] like Figure 4 As shown, the processing device 301 may include one or more processing units 3011, which store various programs and data required for the operation of the electronic device 300. For example, the above data may include Figure 1The index lookup table obtained in . The processing units are connected to each other. The activation buffer 3012 and the weight buffer 3013 are electrically connected to the one or more processing units 3011. The one or more processing units 3011 are electrically connected to the accumulator 3014. The accumulator 3014 is electrically connected to one or more encoders 3015. The one or more encoders 3015 are electrically connected to the output buffer 3016. The information stored in the activation buffer 3012 and the weight buffer 3013 can be input to the one or more processing units 3011. Then, the one or more processing units 3011 can input the generated data to the accumulator 3014. Then, the accumulator 3014 can input the generated data to the one or more encoders 3015. Finally, the one or more encoders 3015 can input the generated data to the output buffer 3016. Among them, the activation buffer 3012 can be used to store various activation information. The activation buffer 3012 can input the stored activation information to the one or more processing units 3011. The weight buffer 3013 may be used to store various weight information. The weight buffer 3013 may input the stored various weight information into the one or more processing units 3011. The one or more processing units 3011 may be used to store the index lookup table, and receive the data input by the activation buffer 3012 and the weight buffer 3013, and obtain various output data after the input data passes through the index lookup table. The one or more processing units 3011 may input the obtained various output data into the accumulator 3014. The accumulator 3014 may perform cumulative calculation on the obtained various output data to obtain accumulated data. The accumulator 3014 may input the accumulated data into the one or more encoders 3015. The one or more encoders 3015 may perform encoding processing on the obtained accumulated data to obtain encoded data. The one or more encoders 3015 may input the encoded data into the output buffer 3016. The output buffer 3016 may store the obtained encoded data.

[0106] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: the processing device 301 included in the electronic device 300 of some embodiments of the present disclosure reduces the computing resources required for model calculation. Specifically, the reason for the large number of computing resources required for model calculation is that the original calculation is to obtain the data to be output by the processing device 301 independently, resulting in the model requiring large computing resources to obtain the data to be output. Based on this, the present disclosure includes one or more processing units 3011, an activation buffer 3012, a weight buffer 3013, an accumulator 3014, one or more encoders 3015 and an output buffer 3016 by using the above-mentioned processing device 301. Thus, the computing resources required for the model can be reduced. Also because the index lookup table used is stored in one or more processing units 3011, in the process of actually obtaining the output data, only the input data and the index lookup table are needed to find the corresponding output data, so that the data that originally needs to be calculated is converted into lookup data, so the computing resources required for model calculation can be reduced.

[0107] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0108] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A model indexing method for deep neural network model compression, include: Receive target neural network model information, wherein the target neural network model information includes information of each neural network layer, each neural network layer information in the each neural network layer information includes activation layer information and weight layer information, the activation layer information includes each activation information, and the weight layer information includes each weight information; Based on the target neural network model information, generating each initial number of centroids; Based on the target neural network model information and the initial number of centroids, generating each centroid information group and compressed standard model information; Generate an index pair group set based on the target neural network model information and each centroid information group; Sending the index pair code set to a display terminal, so that the display terminal displays the received index pair code set; Storing the index pair code set and the compression standard model information in a data storage server; Performing data conversion processing on the index pair code group set to obtain an index lookup table; The index lookup table is stored in a hardware storage structure.

2. The method according to claim 1, in, The generating of each initial number of centroids based on the target neural network model information comprises: For each neural network layer information in the target neural network model information, perform the following steps: Based on the neural network layer information, a sample distribution range is generated, wherein the sample distribution range includes an activation sample distribution range and a weight sample distribution range; Generate an initial number of activation centroids based on the sample distribution range and activation layer information included in the neural network layer information; Generate an initial number of weight centroids based on the sample distribution range and the weight layer included in the neural network layer information; The initial number of activated centroids and the initial number of weighted centroids are integrated to obtain an initial number of centroids.

3. The method according to claim 2, in, The generating the initial number of activated centroids based on the sample distribution range and the activation layer information included in the neural network layer information comprises: Based on the first preset number of centroids, the sample distribution range and the preset number of sampling times, generating first expected clustering information and each first expected distance information; Generate first centroid clustering information based on a first preset number of centroids and activation layer information included in the neural network layer information; Generate first clustering information based on the first expected clustering information and the first centroid clustering information; Generate a first standard deviation based on the first expected clustering information, the respective first expected distance information and the preset sampling times; For the activation layer information included in the neural network layer information, the following centroid loop steps are performed: Based on the second preset number of centroids, the sample distribution range and the preset number of sampling times, generating second expected clustering information and each second expected distance information; Generate second centroid clustering information based on a second preset number of centroids and activation layer information included in the neural network layer information; generating second clustering information based on the second expected clustering information and the second centroid clustering information; Generate a second standard deviation based on the second expected clustering information, the respective second expected distance information and the preset sampling times; In response to determining that the first clustering information, the second clustering information, and the second standard deviation do not satisfy the preset difference condition, determining the second clustering information as the first clustering information to update the first clustering information, determining the second preset number of centroids as the first preset number of centroids to update the first preset number of centroids, and adjusting the second preset number of centroids, and performing the centroid loop step again using the adjusted second preset number of centroids; In response to determining that the first clustering information, the second clustering information, and the second standard deviation satisfy the preset difference condition, a first preset number of centroids is determined as the initial activation number of centroids.

4. The method according to claim 3, in, The generating of first expected clustering information and each first expected distance information based on the first preset number of centroids, the sample distribution range and the preset number of sampling times includes: Based on the preset sampling times, perform the following steps: Based on the sample distribution range, generate each random sample information; Generate each sample cluster based on each random sample information and the first preset number of centroids; Based on the information of each random sample and the information of each sample cluster, generating distance information of each sample; Determine the sum of the distance information of each sample as the first expected distance information; Based on the determined first expected distance information and the preset sampling times, first expected clustering information is generated.

5. The method according to claim 1, in, The step of generating an index pair group set based on the target neural network model information and each centroid information group includes: For each neural network layer information in the target neural network model information, perform the following steps: Determine the centroid information group corresponding to the neural network layer information as the centroid information group to be indexed; Sorting and encoding the to-be-indexed centroid information group to obtain an indexed centroid information group; Performing integer coding processing on each activation information in the activation layer information included in the neural network layer information to obtain each integer activation information; Performing integer coding processing on each piece of weight information in the weight layer information included in the neural network layer information to obtain each piece of integer weight information; Generate an index pair group based on the respective integer activation information and the respective integer weight information; The obtained index pair groups are determined as an index pair group set.

6. The method according to claim 1, in, After generating the index pair group set based on the target neural network model information and the respective centroid information groups, the method further includes: In response to detecting a test data set input by a target user, generating a test accuracy rate based on the compression standard model information and the test data set; In response to determining that the test accuracy satisfies a preset result condition, sending first preset result information to a display terminal, so that the display terminal displays the received first preset result information; In response to determining that the test accuracy rate does not satisfy the preset result condition, second preset result information is sent to a display terminal, so that the display terminal displays the received second preset result information.

7. A model indexing device for deep neural network model compression, include: A receiving unit is configured to receive target neural network model information, wherein the target neural network model information includes information of various neural network layers, each neural network layer information in the various neural network layer information includes activation layer information and weight layer information, the activation layer information includes various activation information, and the weight layer information includes various weight information; A first generating unit is configured to generate each initial number of centroids based on the target neural network model information; A second generating unit is configured to generate each centroid information group and compressed standard model information based on the target neural network model information and the each initial centroid number; A third generating unit is configured to generate an index pair group set based on the target neural network model information and the respective centroid information groups; a sending unit, configured to send the index pair code set to a display terminal, so that the display terminal displays the received index pair code set; A first storage unit is configured to store the index pair code set and the compression standard model information in a data storage server; A data conversion processing unit is configured to perform data conversion processing on the index pair code set to obtain an index lookup table; The second storage unit is configured to store the index lookup table in a hardware storage structure.

8. An electronic device, include: One or more processors, wherein each of the one or more processors comprises one or more processing units, an activation buffer, a weight buffer, an accumulator, one or more encoders, and an output buffer; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer readable medium having a computer program stored thereon, in, When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images