Data compression method and apparatus, data processing method and apparatus, and electronic device and medium

Abstract

Provided in the present disclosure are a data compression method and apparatus, a data processing method and apparatus, and an electronic device and a medium. The data compression method comprises: respectively splitting m pieces of first data to be stored, so as to obtain m pieces of second data and m pieces of third data; and compressing and encoding the m pieces of second data to obtain fourth data corresponding to the m pieces of second data.

Description

Data compression method, data processing method and device, electronic equipment and medium TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and particularly relates to a data compression method, a data processing method and device, an electronic equipment and a medium. BACKGROUND

[0002] A neural network is generally composed of multiple network layers, each network layer usually contains multiple neurons, and the neurons in each network layer are connected with the neurons of the previous layer or input data. In the process of forward propagation, each network layer usually needs to read the multiple activation values output by the neurons in the previous network layer stored in the storage space, and perform calculation based on the multiple activation values and the weights of the neurons in the network layer to obtain the output data of the network layer. SUMMARY

[0003] The present disclosure provides a data compression method, a data processing method and device, an electronic equipment and a medium.

[0004] In a first aspect, the present disclosure provides a data compression method, which comprises:

[0005] m first data to be stored are respectively split to obtain m second data and m third data, wherein the value distribution of the m second data meets a preset condition, the value distribution of the m third data does not meet the preset condition, and the preset condition comprises that the entropy value of the value distribution of a numerical value is less than a preset threshold, and m>1.

[0006] The m second data are compressed and encoded to obtain fourth data corresponding to the m second data, wherein the data amount of the fourth data is less than the data amount of the m second data.

[0007] In a second aspect, the present disclosure provides a data processing method, which comprises:

[0008] In the case that a neural network performs a target processing task, fourth data is read from a first storage space and m third data are read from a second storage space for a first network layer of the neural network, wherein the fourth data and the m third data are obtained after m first data are stored according to the data compression method of the first aspect, and the m first data are m activation values output by the first network layer in the process of forward propagation.

[0009] The fourth data is decompressed to obtain m second data.

[0010] generate the m first data according to the m second data and the m third data;

[0011] take the m first data as input data of a second network layer of the neural network, process the input data based on the second network layer, and obtain output data of the second network layer.

[0012] In a third aspect, the present disclosure provides a data compression device, which comprises:

[0013] a data splitting module configured to split m first data to be stored respectively to obtain m second data and m third data, wherein a value distribution of the m second data meets a preset condition, a value distribution of the m third data does not meet the preset condition, and the preset condition comprises that an entropy value of the value distribution of the numerical value is less than a preset threshold, and m>1;

[0014] a data encoding module configured to compress and encode the m second data to obtain fourth data corresponding to the m second data, wherein a data amount of the fourth data is less than a data amount of the m second data.

[0015] In a fourth aspect, the present disclosure provides a data processing device, which comprises:

[0016] a data reading module configured to read, in a case where a neural network executes a target processing task, fourth data from a first storage space and m third data from a second storage space for a first network layer of the neural network, wherein the fourth data and the m third data are obtained after m first data are stored according to the data compression method of the first aspect, and the m first data are m activation values output by the first network layer in a forward propagation process;

[0017] a data decompression module configured to decompress the fourth data to obtain m second data;

[0018] a data generation module configured to generate the m first data according to the m second data and the m third data;

[0019] a data processing module configured to take the m first data as input data of a second network layer of the neural network, process the input data based on the second network layer, and obtain output data of the second network layer.

[0020] In a fifth aspect, the present disclosure provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the data compression method of the first aspect or the data processing method of the second aspect.

[0021] In a sixth aspect, the present disclosure provides a computer-readable storage medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements the data compression method of the first aspect or the data processing method of the second aspect.

[0022] In a seventh aspect, the present disclosure provides a computer program product comprising computer readable code, or a non-transitory computer-readable storage medium carrying computer readable code, wherein when the computer readable code is run in a processor of an electronic device, the processor in the electronic device performs the data compression method or the data processing method described above.

[0023] The embodiments provided by the present disclosure consider that, for m first data, the overall value distribution thereof does not satisfy a preset condition for data compression, which may be, for example, that the entropy value of the value distribution of the data is less than a preset threshold. However, in the case of locally splitting the data, the split data may satisfy the preset condition for data compression. Therefore, by splitting each first data respectively, the data satisfying the preset condition in the value distribution of the first data is obtained as second data, and the data not satisfying the preset condition is obtained as third data. Then, by compressing and encoding the second data satisfying the preset condition, the part of the m first data whose value distribution can be compressed is compressed, for example, the part of the m first data whose value distribution has an entropy value less than the preset threshold is compressed, thereby obtaining fourth data whose data amount is less than that of the original m second data. In this way, compared with the storage space required for directly storing the m first data, by storing the compressed fourth data and the m third data respectively, the excessive occupation of the storage space can be reduced, and the utilization efficiency of the storage space is improved.

[0024] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0025] The accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and are used to explain the present disclosure and are not intended to limit the present disclosure. The above and other features and advantages of the present disclosure will become more apparent from the detailed description in conjunction with the accompanying drawings, in which:

[0026] Fig. 1 is a flow chart of a data compression method according to an embodiment of the present disclosure;

[0027] Fig. 2 is a flow chart of data splitting processing according to an embodiment of the present disclosure;

[0028] Fig. 3 is a diagram of data bits of a floating point type data according to an embodiment of the present disclosure;

[0029] Fig. 4 is a flow chart of compression encoding processing according to an embodiment of the present disclosure;

[0030] Fig. 5a is a first diagram of difference processing according to an embodiment of the present disclosure;

[0031] Fig. 5b is a second diagram of difference processing according to an embodiment of the present disclosure;

[0032] Fig. 6 is a flow chart of a data processing method according to an embodiment of the present disclosure;

[0033] Fig. 7 is a block diagram of a data storage device according to an embodiment of the present disclosure;

[0034] Fig. 8 is a block diagram of a data processing device according to an embodiment of the present disclosure;

[0035] Fig. 9 is a block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] In order for those skilled in the art to better understand the technical solutions of the present disclosure, the following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to help understanding, and should be considered only as exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, for the sake of clarity and conciseness, the description below omits the description of well-known functions and structures.

[0037] In the case of no conflict, each embodiment of the present disclosure and each feature in the embodiments can be combined with each other.

[0038] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0039] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. "Coupled" or "connected" or similar terms are not restricted to physical or mechanical connections or associations, but can also include electrical connections, whether direct or indirect.

[0040] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted in an overly literal or overly formal sense unless expressly so defined herein.

[0041] In the case of an electronic device performing a computing task, for a set of data, the set of data can usually be directly stored, and when the set of data is needed, the set of data is read from the storage space for use. In the case of a small amount of data or sufficient hardware resources, this direct storage of data does not bring a large load to the storage. However, in the case of a large amount of data or insufficient hardware resources, this direct storage of data usually brings a large load to the storage, affecting the utilization efficiency of the storage space. For example, in the case of a neural network performing a computing task, in the process of forward propagation, each network layer can output a plurality of activation values. If the plurality of activation values of each network layer are directly stored using a direct storage method, a large amount of storage space is usually occupied, affecting the utilization efficiency of the storage space.

[0042] In the related art, in order to reduce the excessive occupation of a set of data to the storage space, a compression algorithm is usually used to compress the data to reduce the amount of data to be stored.

[0043] For example, based on the LZW (Lempel-Ziv-Welch Encoding) encoding algorithm, the algorithm establishes a character conversion table to map an input string to a variable-length code word according to the character conversion table. In the process of encoding data based on the LZW encoding algorithm, if the current character is in the character conversion table, the corresponding encoding of the character in the dictionary is directly used. If the current character is not in the character conversion table, the character is added to the dictionary, and a special encoding is used to represent it. In this way, the data is compressed to reduce the amount of data to be stored.

[0044] Another example is a compression algorithm based on Huffman coding, which uses a variable-length coding table to encode source symbols. This coding table is constructed according to the frequency of the symbols, and symbols with higher frequencies usually use shorter codes. The construction of Huffman coding usually includes the process of constructing a Huffman tree, in which symbols with lower frequencies are located at deeper positions in the tree, and symbols with higher frequencies are located at shallower positions in the tree, so that the average length expectation of the string after coding is reduced to achieve data compression.

[0045] Of course, there are other compression algorithms in the related art for compressing a set of data, but as shown in the above simple description of the LZW encoding algorithm and the Huffman encoding algorithm, such algorithms usually require that the set of data to be stored satisfy certain rules. For example, the LZW encoding algorithm requires that there are more strings in the set of data that can be randomly encoded, and if the characters in the set of data are random, the LZW encoding algorithm may not be able to reduce the data amount. When a set of data is encoded and compressed based on the Huffman encoding algorithm, it also has requirements for the frequency, i.e. the frequency of characters in the set of data, and if the occurrence of characters in the set of data is completely random, it may also not be able to reduce the data amount.

[0046] Therefore, in the data compression method provided in the embodiments of the present disclosure, considering that the overall value distribution of the m first data does not satisfy the preset condition for data compression, which can be, for example, that the entropy value of the value distribution of the data is less than a preset threshold, however, in the case of local splitting of the data, the split data can satisfy the preset condition for data compression, therefore, by splitting each first data respectively, the data in the first data whose value distribution satisfies the preset condition is obtained as second data and the data that does not satisfy the preset condition is obtained as third data, then, by compressing and encoding the second data that satisfies the preset condition, the part of the m first data whose value distribution can be compressed can be compressed, for example, the part of the m first data whose value distribution entropy value is less than the preset threshold can be compressed, thereby obtaining fourth data whose data amount is less than the data amount of the original m second data. In this way, compared with the storage space required for directly storing the m first data, by storing the compressed fourth data and the m third data respectively, the excessive occupation of storage space can be reduced, and the utilization efficiency of storage space can be improved.

[0047] The data compression method and the data processing method according to the embodiments of the present disclosure can be executed by an electronic device such as a terminal device or a server, and the terminal device can be a vehicle-mounted device, a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The method can be implemented by a processor invoking computer readable program instructions stored in a memory. Alternatively, the method can be executed by a server.

[0048] FIG. 1 is a flowchart of a data compression method according to an embodiment of the present disclosure. Referring to FIG. 1, the method includes:

[0049] In step S11, m first data to be stored are respectively split to obtain m second data and m third data.

[0050] The value distribution of the m second data satisfies a preset condition, the value distribution of the m third data does not satisfy the preset condition, and m > 1. The preset condition may, for example, include that the entropy value of the value distribution of the numerical value is less than a preset threshold.

[0051] In the embodiments of the present disclosure, the m first data can be a group of data generated or obtained in the process of executing a computing task by an electronic device. For example, the first data can be an activation value in the process of forward propagation of a neural network. The neural network can be used to execute a target processing task, which can be any one of an image processing task, a speech processing task, a text processing task, and a video processing task. Of course, the first data can also be a weight value of a neuron of each network layer of the neural network, or can be set as needed, which is not specially limited here.

[0052] In the embodiments of the present disclosure, the first data can be any type of data including at least one data bit. It should be noted that the data bit in the embodiments of the present disclosure can be used to represent a part of data in a single data.

[0053] For example, in the case that the first data is floating point value data, it can include an exponent bit, a significant digit bit (also referred to as a mantissa bit), and a sign bit, the exponent bit is used to represent the value of the exponent part of the floating point value data, the sign bit is used to represent the sign part of the floating point value data, and the significant digit bit is used to represent the significant digit part of the floating point value data. For example, for the floating point value data "-9.625", the sign bit is "1", the exponent bit is "10000010", and the significant digit bit is "0011010000000000000". Of course, the first data can also be data composed of multiple data fields, each data field can be regarded as a data bit of the data. For example, the first data can be data composed of two data fields of "number" and "name". For "0010user01", the "number" data bit is "0010", and the "name" data bit is "user01".

[0054] It should be noted that, in the embodiments of the present disclosure, the preset condition is a condition for judging whether the value distribution of the numerical value meets the data compression, which can be, for example, that the entropy value of the value distribution of the numerical value is less than a preset threshold, and the preset threshold can be set as needed. For another example, the preset condition can be that the occurrence frequency or probability of a string of a preset length in the numerical value is greater than a preset threshold, and the preset condition can also be set as needed in actual implementation, which is not specially limited here.

[0055] In step S12, the m second data are compressed and encoded to obtain fourth data corresponding to the m second data, wherein the data amount of the fourth data is less than the data amount of the m second data.

[0056] In an optional implementation, when the m first data to be stored are respectively split, the following manner is used: whether the value distribution of the numerical value in different data bits of the m first data to be stored meets the preset condition is judged, and the m first data to be stored are split according to the judgment result. For example, if the judgment result is no, it indicates that the m first data to be stored do not meet the condition of data compression, and accordingly, the m first data to be stored are split to obtain m second data and m third data, so that the value distribution of the m second data meets the preset condition, and the m second data can be compressed. As can be seen, in the case that the value distribution of the numerical value in different data bits of the m first data to be stored does not meet the preset condition, the m first data to be stored are split, and different processing manners are performed on the second data and the third data after the splitting, so as to reduce the storage amount.

[0057] In step S13, the fourth data is stored in the first storage space, and the m third data is stored in the second storage space. The step S13 is an optional step, and in other embodiments, the step S13 can be omitted.

[0058] In the embodiments of the present disclosure, the first storage space and the second storage space can be any two different storage spaces, and the embodiments of the present disclosure do not make special limitations on this.

[0059] It can be seen that, according to the embodiments of the present disclosure, by respectively splitting each first data, the data whose value distribution satisfies the preset condition, for example, the entropy value of the value distribution of the numerical value is less than the preset threshold, is obtained as the second data, and the data that does not satisfy the preset condition is obtained as the third data. Then, by compressing and encoding the second data that satisfies the preset condition, the part of the m first data whose value distribution can be compressed is compressed to obtain the fourth data whose data amount is less than that of the original m second data. In this way, compared with the storage space required by directly storing the m first data, by respectively storing the compressed fourth data and the m third data, the excessive occupation of the storage space can be reduced, and the utilization efficiency of the storage space is improved.

[0060] FIG. 2 is a flowchart of a data splitting process provided by the embodiments of the present disclosure. In some possible implementation manners, the first data can include a plurality of data bits; referring to FIG. 2, in this implementation manner, in step S11, the splitting of the m first data to be stored respectively to obtain the m second data and the m third data can include:

[0061] In step S111, for any first data, the data bit of the first data that satisfies the preset condition is taken as the second data corresponding to the first data.

[0062] In step S112, the data bit of the first data other than the second data is taken as the third data corresponding to the first data.

[0063] In information theory, entropy can be used to describe the uncertainty of a source. The entropy value can be any one of information entropy, cross entropy, relative entropy, conditional entropy, etc. In the embodiments of the present disclosure, the entropy value of the value distribution of the numerical value can be the information entropy of a plurality of numerical values, or can be set as needed, which is not specially limited here.

[0064] That is, in the embodiments of the present disclosure, for different data bits in the m first data, the data in different categories of data bits can be split, and then the entropy value of the value distribution of the data in each category of data bits is calculated respectively, and compared with a preset threshold. If the entropy value of the value distribution of the data in a certain category of data bits is low, it means that the data is stable, and the value distribution is usually within a certain data range, so the data in this category of data bits can be considered as the second data.

[0065] Taking m as 10 and the first data including data bit 1 and data bit 2 as an example, the values of data bit 1 can be split from the 10 data to obtain 10 sub-data 1 corresponding to the 10 data, and the values of data bit 2 can be split to obtain 10 sub-data 2 corresponding to the 10 data. If the entropy value of the value distribution of the 10 sub-data 1 is e1, the entropy value of the value distribution of the 10 sub-data 2 is e2, and e1 is less than the preset threshold and e2 is greater than the preset threshold, the 10 sub-data 1 can be regarded as the second data in the embodiments of the present disclosure, and the 10 sub-data 2 can be regarded as the third data in the embodiments of the present disclosure.

[0066] Since the entropy value of the 10 sub-data 1 is low, that is, the value distribution of the data is usually within a certain data range, the 10 sub-data 1 can be compressed and encoded, for example, using the LZW encoding algorithm or the Huffman encoding algorithm in related technologies to encode this part of data to obtain fourth data with a data amount much smaller than the 10 sub-data 1. Then by storing the fourth data and the 10 sub-data 2 respectively, the data storage amount can be reduced, and the storage space can be effectively utilized.

[0067] In some possible implementation manners, the m first data can be floating-point data, and the data bits can include exponent bits. In this implementation manner, the step S111 of taking the data bit of the first data that meets the preset condition as the second data corresponding to the first data includes: taking the exponent bit of the first data as the second data. The step S112 of taking the data bit of the first data except the second data as the third data corresponding to the first data includes: taking the data bit of the first data except the exponent bit as the third data.

[0068] Specifically, refer to FIG. 3, which is a data bit diagram of floating point type data provided by the embodiment of the present disclosure. As shown in FIG. 3, for the floating point type data "-9.625" represented in decimal, when it is stored in a computer, it is usually stored in the format of "sign bit", "exponent bit" and "significant digit bit", so "1" can be set in the sign bit to indicate that the sign bit of "-9.625" is negative, "10000010" can be set in the exponent bit to indicate that the exponent bit is 3, and "0011010000000000000" can be set in the significant digit bit to indicate the significant digit bit.

[0069] In the case of data being floating point type data, the entropy values of the value distribution of the exponent bit and the significant data bit are often different, that is, in a group of floating point type data, the entropy value of the significant digit bit is usually high, and the entropy value of the exponent bit is usually low. For example, in the case of neural network performing a computing task, for input data 0, multiple activation values generated by the same network layer in the forward propagation process are often within a certain data range, which also leads to the same data or data with small difference in the exponent bit, while the significant digit bit is usually more diverse, for example, for 10 activation values: "-9.625, -10.245, -8.176, -7.121, -9.963, -6.315, -8.425, -10.831, -9.462, -10.429", the value of the exponent bit is 3 or 4, while the significant digit bit is more random, therefore, in the embodiment of the present disclosure, in the case of the m first data being floating point type data, the exponent bit of each first data can be taken as the second data, and the data bit of each first data except the exponent bit can be taken as the third data.

[0070] For example, for the above 10 activation values: "-9.625, -10.245, -8.176, -7.121, -9.963, -6.315, -8.425, -10.831, -9.462, -10.429", the exponent bit of each data can be considered as its second data, and other data bits such as sign bit and significant digit bit can be considered as its third data, in the case of taking the exponent bit of the 10 activation values as its second data, since the value of the obtained 10 second data is 3 or 4, therefore, by compressing the 10 second data into 1 fourth data and storing the fourth data and other third data, the occupation of storage space can be reduced without losing the original data.

[0071] It can be seen that, based on the method provided in the embodiments of the present disclosure, by splitting the plurality of first data based on the data bits of the first data, judging whether the value distribution of the values in different data bits meets the preset condition, and compressing the data bits meeting the preset condition, compared with directly storing the plurality of first data, the excessive occupation of the storage space can be reduced, and the utilization efficiency of the storage space can be improved.

[0072] In some possible implementation ways, in step S12, the m second data are compressed and encoded to obtain fourth data corresponding to the m second data, including: obtaining n reference data, wherein the n reference data include n data selected from the m second data, and 1≤n

[0073] FIG. 4 is a flowchart of a compression and encoding process provided in the embodiments of the present disclosure. Referring to FIG. 4, in some possible implementation ways, the fourth data can be obtained based on the n reference data and the m second data, which can include: in step S121, differentiating the m second data and the n reference data to obtain m difference values; and in step S122, obtaining the fourth data according to the n reference data and the m difference values.

[0074] In some possible implementation ways, the fourth data can be obtained according to the n reference data and the m difference values, which can include: generating a sparse code corresponding to the m second data based on the m difference values, and generating the fourth data based on the n reference data and the sparse code.

[0075] In some possible implementation ways, in step S121, the m difference values can be obtained by differentiating the m second data and the n reference data, which can include: in the case that n is equal to 1, differentiating the m second data and the reference data respectively to obtain the m difference values; or in the case that n is greater than 1, obtaining a mapping relationship between the n reference data and the m second data, and differentiating each second data and a target reference data based on the mapping relationship to obtain the m difference values, wherein the target reference data is a reference data in the n reference data that has a mapping relationship with the second data.

[0076] In the case that n is equal to 1, the reference data can be any one of the m second data. Referring to FIG. 5a, which is a first schematic diagram of the differential processing provided by the embodiments of the present disclosure. As shown in FIG. 5a, in the case that the m first data are 10 activation values: "-9.625, -10.245, -8.176, -7.121, -9.963, -6.315, -8.425, -10.831, -9.462, -10.429", the second data thereof can be respectively represented as "3, 4, 3, 3, 3, 3, 3, 4, 3, 4", if the reference data is 3, then by differentiating the m second data respectively from the reference data, the differential values can be obtained as "0, 1, 0, 0, 0, 0, 0, 1, 0, 1", and based on the 10 differential values, the sparse code "0100100101" can be generated, then in the case that the fourth data is in the form of "reference data + sparse code", the fourth data corresponding to the original 10 first data can be represented as "3+0100000101", compared with the storage space required for storing the original 10 activation values respectively, by storing the fourth data obtained by compressing the 10 second data and the 10 third data corresponding to the 10 activation values, the occupation of the storage space can be significantly reduced.

[0077] In addition, in the case that n is greater than 1, the reference data can be any n second data of the m second data, referring to FIG. 5b, which is a second schematic diagram of the differential processing provided by the embodiments of the present disclosure. As shown in FIG. 5b, in the case that the m first data are 10 activation values: "-9.625, -10.245, -8.176, -7.121, -9.963, -6.315, -8.425, -10.831, -9.462, -10.429", the second data thereof can be respectively represented as "3, 4, 3, 3, 3, 3, 3, 4, 3, 4", the reference data 1 is 3, and the reference data 2 is 4, then for example, the following mapping relationship can be established, the reference data 1 corresponds to the 1st, 3rd to 7th, and 9th second data, and the reference data 2 corresponds to the 2nd, 8th, and 10th second data, then by differentiating the 1st, 3rd to 7th, and 9th second data, the differential value 1 can be obtained as 0, and by differentiating the 2nd, 8th, and 10th second data from the reference data 2, the differential value 2 can be obtained as 0, then based on the differential value 1 and the differential value 2, the sparse code "0000000000" can be generated, thereby based on the two reference data and the sparse code, the fourth data "3+0100000101" can be generated, of course, in this case, the fourth data can also include the mapping relationship described above, or the mapping relationship can be stored as a default configuration, which is not specially limited here.

[0078] It should be noted that the above is only an example, and in some possible implementations, the reference data can also be set as needed when n is equal to 1, for example, it can be the average or median of the m second data, or any numerical value. In addition, when n is greater than 1, the n reference data can also be set as follows: the i-th second data in the m second data is taken as the reference data of the i+1-th second data, where i < m, for example, for the second data "3, 4, 3, 3, 3, 3, 3, 4, 3, 4" above, the first second data can be taken as the reference data of the second second data, the second second data can be taken as the reference data of the third second data, and so on, the ninth second data can be taken as the reference data of the tenth second data, that is, each second data is differentiated with its previous value, and sparse coding is generated based on the obtained difference value, which is not specially limited here.

[0079] It can be seen that, based on the data compression method provided by the embodiments of the present disclosure, by respectively splitting each first data, the data whose value distribution in the first data meets the preset condition is obtained as the second data and the data that does not meet the preset condition is obtained as the third data, then by compressing and encoding the second data that meets the preset condition, the part of the m first data whose value distribution can be compressed can be compressed to obtain the fourth data whose data amount is less than that of the original m second data. In this way, compared with the storage space required to directly store the m first data, by respectively storing the compressed fourth data and the m third data, the excessive occupation of the storage space can be reduced, and the utilization efficiency of the storage space can be improved.

[0080] Corresponding to the data compression method provided by the embodiments of the present disclosure, the embodiments of the present disclosure also provide a data processing method, and FIG. 6 is a flowchart of a data processing method provided by an embodiment of the present disclosure. Referring to FIG. 6, the method comprises:

[0081] In step S61, when the neural network performs a target processing task, the fourth data is read from the first storage space and the m third data is read from the second storage space for the first network layer of the neural network.

[0082] Among them, the fourth data and the m third data are obtained after the m first data is stored according to the data compression method of the embodiments of the present disclosure, and the m first data is the m activation value output by the first network layer in the forward propagation process.

[0083] In step S62, the fourth data is decompressed to obtain the m second data.

[0084] In step S63, the m first data is generated according to the m second data and the m third data.

[0085] In step S64, the m first data are taken as input data of a second network layer of the neural network, the input data are processed based on the second network layer to obtain output data of the second network layer.

[0086] The neural network can be a neural network of any structure, and the present disclosure does not make special limitations on this.

[0087] The target processing task can be any one of an image processing task, a speech processing task, a text processing task, and a video processing task, for example, can be a text classification task in a text processing task.

[0088] In this implementation, in the case where the neural network performs the target processing task, to improve the utilization efficiency of the storage space, the m activation values generated by the first network layer can generate corresponding fourth data and third data based on the data compression method described above, so as to reduce the data storage amount, the second network layer after the first network layer can read the fourth data and the third data, restore the fourth data to m second data, and then restore the original m activation values based on the second data and the third data, so as to take the m activation values as input data for processing by the second network layer to obtain output data, based on this method, the neural network can obtain the final output result.

[0089] It can be seen that, based on the data processing method provided by the present disclosure, in the case where the neural network performs the target processing task, the activation values generated by each network layer in the forward propagation process of the neural network are stored based on the data compression method described above, which can reduce the occupation of the storage space and improve the utilization efficiency of the storage space.

[0090] It can be understood that, the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form combined embodiments without violating the principle logic. Due to the limited length, the present disclosure will not be described again. It can be understood by those skilled in the art that the specific execution order of each step in the above-mentioned method should be determined according to its function and possible internal logic.

[0091] In addition, the present disclosure also provides a data storage device, a data processing device, an electronic device, and a computer readable storage medium, which can be used to implement any one of the data compression methods or data processing methods provided by the present disclosure. The corresponding technical solutions and descriptions are described in the method part and will not be described again.

[0092] FIG. 7 is a block diagram of a data storage device provided by an embodiment of the present disclosure.

[0093] Referring to FIG7, an embodiment of this disclosure provides a data compression device, which includes a data splitting module 71, a data encoding module 72, and a data storage module 73.

[0094] The data splitting module 71 is used to split the m first data to be stored into m second data and m third data. The value distribution of the m second data satisfies the preset conditions, while the value distribution of the m third data does not satisfy the preset conditions. The preset conditions include: the entropy value of the value distribution is less than a preset threshold, and m > 1.

[0095] The data encoding module 72 is used to compress and encode m second data to obtain fourth data corresponding to the m second data, wherein the amount of data of the fourth data is less than the amount of data of the m second data;

[0096] The data storage module 73 is used to store the fourth data in the first storage space and store m third data in the second storage space.

[0097] The data storage module 73 is an optional module, and in other embodiments, the data storage module 73 may be omitted.

[0098] In some possible implementations, when the data splitting module 71 splits the m first data to be stored, it needs to determine whether the value distribution of different data bits in the m first data to be stored meets a preset condition, and split according to the determination result; wherein, the first data includes activation values ​​in the forward propagation process of the neural network. In some possible implementations, the first data includes multiple data bits; when the data splitting module 71 splits the m first data to be stored to obtain m second data and m third data, it can be used to: for any first data, take the data bits of the first data that meet the preset condition as the second data corresponding to the first data; take the data bits of the first data other than the second data as the third data corresponding to the first data.

[0099] In some possible implementations, the m first data include floating-point data, and the data bits include exponent bits; when the data splitting module 71 uses the data bits of the first data that meet the preset conditions as the second data corresponding to the first data, it can be used to: use the exponent bits of the first data as the second data; when using the data bits of the first data other than the second data as the third data corresponding to the first data, it can be used to: use the data bits of the first data other than the exponent bits as the third data.

[0100] In some possible implementation manners, when the data encoding module 72 performs compression encoding on the m second data to obtain the fourth data corresponding to the m second data, the data encoding module 72 can be configured to: obtain n reference data, where the n reference data includes n data selected from the m second data, and 1≤n

[0101] In some possible implementation manners, when the data encoding module 72 performs compression encoding on the m second data to obtain the fourth data corresponding to the m second data, the data encoding module 72 can be configured to: obtain n reference data, where the n reference data includes n data selected from the m second data, and 1≤n

[0102] In some possible implementation manners, when the data encoding module 72 performs compression encoding on the m second data to obtain the fourth data corresponding to the m second data, the data encoding module 72 can be configured to: obtain n reference data, where the n reference data includes n data selected from the m second data, and 1≤n

[0103] FIG. 8 is a block diagram of a data processing apparatus provided by an embodiment of the present disclosure.

[0104] Referring to FIG. 8, the data processing apparatus provided by an embodiment of the present disclosure includes a data reading module 81, a data decompression module 82, a data generation module 83, and a data processing module 84.

[0105] The data reading module 81 is configured to, in a case where a neural network performs a target processing task, read, from a first storage space, fourth data corresponding to a first network layer of the neural network, and read, from a second storage space, m third data, where the fourth data and the m third data are obtained after m first data are stored according to a data compression method of an embodiment of the present disclosure, and the m first data are m activation values output by the first network layer in a forward propagation process.

[0106] The data decompression module 82 is configured to perform decompression processing on the fourth data to obtain m second data.

[0107] The data generation module 83 is configured to generate the m first data according to the m second data and the m third data.

[0108] The data processing module 84 is configured to take the m first data as input data of a second network layer of the neural network, process the input data based on the second network layer, and obtain output data of the second network layer.

[0109] FIG. 9 is a block diagram of an electronic device according to an embodiment of the present disclosure.

[0110] Referring to FIG. 9, the electronic device according to an embodiment of the present disclosure includes at least one processor 901, at least one memory 902, and one or more I / O interfaces 903 connected between the processor 901 and the memory 902. The memory 902 stores one or more computer programs executable by the at least one processor 901. The one or more computer programs are executed by the at least one processor 901 to enable the at least one processor 901 to perform the data compression method or the data processing method described above.

[0111] In some embodiments, the electronic device can be a brain-like chip. Since the brain-like chip can use vectorized computing, it needs to load parameters such as weight information of a neural network model from an external memory such as a double data rate (DDR) synchronous dynamic random access memory. Therefore, the batch processing operation is more efficient according to an embodiment of the present disclosure.

[0112] An embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor / processing core, implements the data compression method or the data processing method described above. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0113] An embodiment of the present disclosure further provides a computer program product including computer-readable code or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the data compression method or the data processing method described above.

[0114] Those of ordinary skill in the art will realize and understand that all or some of the steps in the methods disclosed above and the functional modules / units in the systems and devices can be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementation, the division between the functional modules / units referred to in the above description does not necessarily correspond to the division of physical components; for example, one physical component can have multiple functions, or one function or step can be performed by several physical components in cooperation. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software can be distributed on computer readable storage media, which can include computer storage media (or non-transitory media) and communication media (or transitory media).

[0115] As is well known to those of ordinary skill in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable program instructions, data structures, program modules or other data. Computer storage media include, but are not limited to, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by a computer. Further, it should be understood by those of ordinary skill in the art that communication media typically embodies computer readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.

[0116] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0117] Computer readable program instructions for carrying out operations of the present disclosure can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or any combination of one or more of the above in any combination of one or more programming languages including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

[0118] The computer program product described herein can be embodied in a specific manner by hardware, software, or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium. In another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), and the like.

[0119] The various aspects of the present disclosure are described herein with reference to flow diagrams and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present disclosure. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer readable program instructions.

[0120] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer- readable storage medium having no data storage cycles that change state. The instructions can be executed by one or more processors of a computer, other programmable data processing apparatus, or other devices to produce a computer-implemented process such that the instructions which execute via the one or more processors of the computer or other programmable data processing devices create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0121] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0122] The flow and block diagrams in the drawings show architectural, functional, and operational aspects of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow and block diagrams can represent a module, a segment, or a portion of instructions which comprise one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may

[0123] Example embodiments have been disclosed and, although a specific terminology is employed, it is merely for the convenience of the reader and is not intended to be limiting. In some instances, specific details have been included for the purpose of providing a thorough understanding of the embodiments presented, but it will be apparent to those skilled in the art that embodiments of the application can be practiced without these specific details. In some instances, features, characteristics and / or elements described in connection with a particular embodiment can be used in conjunction with, or switched to, features, characteristics and / or elements described in connection with other embodiments, unless otherwise explicitly stated. Accordingly, it will be understood that various changes in form and details can be made without departing from the scope of the disclosure as set forth in the appended claims.

Claims

1. A data compression method wherein, The method comprises: splitting m first data to be stored respectively to obtain m second data and m third data, wherein a value distribution of the m second data meets a preset condition, a value distribution of the m third data does not meet the preset condition, the preset condition comprises that an entropy value of the value distribution of the numerical value is less than a preset threshold, and m>1; compressively encoding the m second data to obtain fourth data corresponding to the m second data, wherein a data amount of the fourth data is less than a data amount of the m second data.

2. The method of claim 1, wherein, The splitting of the m first data to be stored respectively comprises judging whether a value distribution of a numerical value in different data bits of the m first data to be stored meets a preset condition, and splitting according to a judgment result; wherein the first data comprises activation values in a neural network forward propagation process.

3. The method of claim 1, wherein, After obtaining the fourth data corresponding to the m second data, the method further comprises storing the fourth data in a first storage space and storing the m third data in a second storage space.

4. The method of claim 1, wherein, The first data comprises a plurality of data bits; and the splitting of the m first data to be stored respectively to obtain m second data and m third data comprises: for any first data, taking data bits of the first data that meet the preset condition as second data corresponding to the first data; taking data bits of the first data other than the second data as third data corresponding to the first data.

5. The method of claim 4, wherein, The m first data comprises floating-point data, and the data bits comprise exponent bits; The taking of the data bits of the first data that meet the preset condition as the second data corresponding to the first data comprises taking exponent bits of the first data as the second data; The taking of the data bits of the first data other than the second data as the third data corresponding to the first data comprises taking data bits of the first data other than the exponent bits as the third data.

6. The method of claim 1, wherein, The compressively encoding of the m second data to obtain fourth data corresponding to the m second data comprises: obtaining n reference data, wherein the n reference data comprises n data selected from the m second data, and 1≤n obtaining the fourth data based on the n reference data and the m second data.

7. The method of claim 6, wherein, The obtaining of the fourth data based on the n reference data and the m second data comprises: differentially processing the m second data and the n reference data to obtain m difference values; obtaining the fourth data according to the n reference data and the m difference values.

8. The method of claim 7, wherein, The differentially processing of the m second data and the n reference data to obtain m difference values comprises: in a case where n is equal to 1, differentially processing the m second data and the reference data respectively to obtain the m difference values; or In a case where n is greater than 1, a mapping relationship between the n reference data and the m second data is obtained, each second data is differentiated with a target reference data based on the mapping relationship, and m difference values are obtained, wherein the target reference data is a reference data in the n reference data that has a mapping relationship with the second data.

9. A data processing method, wherein, Comprising: In a case where a neural network performs a target processing task, a fourth data is read from a first storage space and m third data are read from a second storage space for a first network layer of the neural network, wherein the fourth data and the m third data are obtained after m first data are stored according to the method in any one of claims 1-8, and the m first data are m activation values output by the first network layer in a forward propagation process; The fourth data is decompressed to obtain m second data; The m first data are generated according to the m second data and the m third data; The m first data are taken as input data of a second network layer of the neural network, and output data of the second network layer are obtained by processing the input data based on the second network layer.

10. A data compression device, wherein, Comprising: A data splitting module is configured to split m first data to be stored to obtain m second data and m third data, wherein a value distribution of the m second data meets a preset condition, a value distribution of the m third data does not meet the preset condition, and the preset condition includes that an entropy value of the value distribution of the numerical value is less than a preset threshold, and m>1. A data encoding module is configured to compress and encode the m second data to obtain fourth data corresponding to the m second data, wherein a data amount of the fourth data is less than a data amount of the m second data.

11. An electronic device, comprising: Comprising: At least one processor; And A memory connected in communication with the at least one processor; wherein The memory stores one or more computer programs that can be executed by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to execute the method in any one of claims 1-9.

12. A computer readable storage medium having stored thereon a computer program, wherein, The computer program, when executed by the processor, implements the method in any one of claims 1-9.

13. A computer program product comprising computer readable code, or a non-transitory computer readable storage medium having computer readable code embodied thereon, wherein, When the computer readable code runs in the processor of the electronic device, the processor in the electronic device executes the method in any one of claims 1-9.

Images