Data processing method and device, electronic equipment and storage medium

Abstract

The invention provides a data processing method and device, electronic equipment and a storage medium. The method comprises the steps of obtaining input vector data of a first network layer in a network model; performing first sparse processing on the input vector data and the first target dimension data to obtain first output vector data; second target dimension data are determined according to the first output vector data, and the second target dimension data and the first target dimension data belong to row dimension data or column dimension data in a weight matrix of the first network layer respectively and have no overlapped elements; performing second sparse processing on the input vector data and the second target dimension data to obtain second output vector data; and obtaining target output vector data according to the first output vector data and the second output vector data.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a data processing method and device, an electronic device, and a storage medium. Background Art

[0002] As the scale of parameters contained in the network model becomes larger and larger, the data transmission volume and calculation amount of the network model will also increase, which brings challenges to the resource demand and performance requirements of the network model. It is very important to further reduce the resource consumption of the network model. Summary of the invention

[0003] The present disclosure provides a data processing method and device, an electronic device, and a storage medium.

[0004] In a first aspect, the present disclosure provides a data processing method, the data processing method comprising:

[0005] Get the input vector data of the first network layer in the network model;

[0006] Performing a first sparse processing on the input vector data and the first target dimension data to obtain first output vector data;

[0007] Determine second target dimensional data according to the first output vector data, wherein the second target dimensional data and the first target dimensional data belong to row dimensional data or column dimensional data in the weight matrix of the first network layer respectively, and have no overlapping elements;

[0008] Performing a second sparse processing on the input vector data and the second target dimension data to obtain second output vector data;

[0009] Target output vector data is obtained according to the first output vector data and the second output vector data.

[0010] In a second aspect, the present disclosure provides a data processing device, the data processing device comprising:

[0011] An acquisition module, used to acquire input vector data of the first network layer in the network model;

[0012] A first processing module, configured to perform a first sparse processing on the input vector data and the first target dimension data to obtain first output vector data;

[0013] A determination module, configured to determine second target dimensional data according to the first output vector data, wherein the second target dimensional data and the first target dimensional data respectively belong to row dimensional data or column dimensional data in the weight matrix of the first network layer, and have no overlapping elements;

[0014] A second processing module, used for performing a second sparse processing on the input vector data and the second target dimension data to obtain second output vector data;

[0015] The third processing module is used to obtain target output vector data according to the first output vector data and the second output vector data.

[0016] In a third aspect, the present disclosure provides an electronic device, comprising: multiple processing cores; and an on-chip network configured to exchange data between the multiple processing cores and external data; wherein one or more instructions are stored in one or more of the processing cores, and one or more of the instructions are executed by one or more of the processing cores, so that one or more of the processing cores can execute the above-mentioned data processing method.

[0017] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the above-mentioned data processing method when executed by a processing core.

[0018] In a fifth aspect, the present disclosure provides a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying a computer-readable code. When the computer-readable code runs in a processing core of an electronic device, the processing core in the electronic device executes the above-mentioned data processing method.

[0019] According to the embodiments provided by the present disclosure, when performing calculations on the first network layer in the network model, a first sparse processing can be performed on the input vector data and the first target dimension data in the weight matrix of the first network layer to obtain first output vector data, thereby completing the first stage of sparse calculations. Then, based on the first output vector data, the second target dimension data to be calculated in the weight matrix is ​​determined, and a second sparse processing is performed on the input vector data and the second target dimension data to obtain second output vector data. Finally, based on the first output vector data and the second output vector data, target output vector data is obtained to complete the second stage of sparse calculations. In this way, through sparse processing, more important data can be screened out to participate in the calculation, and only part of the data needs to be loaded and calculated, which can reduce the amount of data loading and calculation, and the output results of multiple stages of sparse processing are integrated to obtain the final target output vector data, which can compensate for the accuracy loss, thereby improving the reasoning speed and performance of the network model.

[0020] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art by describing detailed example embodiments with reference to the accompanying drawings, in which:

[0022] Figure 1 A flowchart of a data processing method provided by an embodiment of the present disclosure;

[0023] Figure 2 This is an example diagram of a data processing method in an embodiment of the present disclosure;

[0024] Figure 3 A block diagram of a data processing device provided in an embodiment of the present disclosure;

[0025] Figure 4 A block diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0027] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0028] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0029] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a" and "the" are also intended to include plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "including" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded. "Connected" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0030] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined as such herein.

[0031] As the scale of parameters contained in network models becomes larger and larger, for example, some large models such as large language models usually have a large number of parameters, the data transmission, computing power, required storage space, etc. of the network models will also increase, which brings challenges to deployment and application. In addition, with the growing demand for privacy protection and timeliness, the demand for local deployment on mobile phones, computers and other devices is gradually becoming more necessary. Therefore, it is very important to further reduce the resource consumption of network models in storage, computing, etc.

[0032] In practice, in reasoning scenarios based on large language models, the amount of computation for each token generated by each step of reasoning in the large language model is usually smaller than the amount of transmission of weight parameters. Therefore, compressing the total storage or bandwidth transmission of the large language model can increase the reasoning speed.

[0033] In addition, large models are mainly based on the transformer architecture, which includes multiple layers, each of which usually includes an attention layer and a forward connection layer. The basic operation is a linear operator, that is, the weight matrix is ​​multiplied by the input vector to obtain the output vector. When reasoning with a large model, the memory loading of the weight matrix is ​​the main factor affecting the reasoning speed. Therefore, reducing the amount of data transmission can speed up the model reasoning speed.

[0034] Therefore, based on this, a processing method for linear operators is proposed in the embodiment of the present disclosure to reduce the amount of data loaded and the amount of calculation, thereby improving the reasoning speed of the network model. According to the data processing method of the embodiment of the present disclosure, for any first network layer in the network model, the input vector data of the first network layer is obtained, and through multi-stage sparse processing, the dimensions of the elements in the input vector data to be calculated and the weight matrix of the first network layer are determined. Only the part of the data to be calculated can be loaded for calculation, which reduces the amount of data loaded and the amount of calculation, reduces resource consumption, thereby speeding up the reasoning speed of the network model and reducing power consumption.

[0035] The data processing method according to the embodiment of the present disclosure can be executed by an electronic device such as a terminal device or a server, and the terminal device can be a vehicle-mounted device, a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The method can be implemented by a processor calling a computer-readable program instruction stored in a memory. Alternatively, the method can be executed by a server.

[0036] It should be noted that in the embodiments of the present disclosure, there is no limitation on the network model, and the network model can be used to perform any of image processing tasks, speech processing tasks, text processing tasks, and video processing tasks. For example, the input vector data of the network model in the embodiments of the present disclosure can include any of the vector representations of images, speech, text, and videos. The output results of network model reasoning can include any of images, speech, text, and videos.

[0037] Figure 1 A flowchart of a data processing method provided by an embodiment of the present disclosure. Figure 1 , the method comprising:

[0038] Step S110: Obtain input vector data of the first network layer in the network model.

[0039] In the embodiments of the present disclosure, mainly for the scenario of task processing based on the network model, the task information can be input into the network model, and then the task information is inferred through each network layer in the network model to obtain the corresponding output result. When the network model is inferred, calculation is performed based on the weight matrix of each network layer in the network model to obtain the output vector for transmission to the next network layer, until the output vector of the last network layer is obtained as the final output result of the reasoning.

[0040] The first network layer in step S110 may be any network layer in the network model.

[0041] Step S120: Perform a first sparse processing on the input vector data and the first target dimension data to obtain first output vector data.

[0042] Among them, the first target dimension data can represent the first data to be involved in the calculation in the weight matrix of the first network layer.

[0043] Regarding step S120, in a possible embodiment of the present disclosure:

[0044] S1: Determine the first value of the element to be calculated in the input vector data and the first target dimension data to be calculated in the weight matrix of the first network layer according to the input vector data and the first loading strategy. The first loading strategy is used to characterize the screening method for the elements in the vector data.

[0045] In the disclosed embodiment, a first sparse processing is performed on the calculation of the network layer in the network model, and elements and dimensions to be involved in the calculation that meet certain conditions are screened out based on a first loading strategy. The first loading strategy can be understood as a way of loading which data to participate in the calculation to achieve sparse processing, thereby reducing the amount of data loaded and the amount of calculation and improving the reasoning speed.

[0046] In a possible embodiment, step S1 includes:

[0047] 1) Determine a first value of an element in the input vector data that complies with a first loading strategy and a first index value in the input vector data corresponding to the first value.

[0048] For example, the first loading strategy is to filter out the M values ​​with the largest absolute values ​​of the elements in the input vector data. For the input vector data, according to the first loading strategy, the M first values ​​with the largest absolute values ​​are determined, and their first index values ​​in the input vector data are obtained. For example, if the input vector data is a 1*4 vector, M is 1, and it is determined that the second value with the largest absolute value in the input vector data is, that is, the first index value is 2.

[0049] 2) According to the first index value, determine the first target dimension data corresponding to the weight matrix of the first network layer, wherein the first value and the first target dimension data are the data to be calculated.

[0050] For example, the weight matrix of the first network layer is a 4*5-dimensional matrix, the first index value is 2, and taking the first target dimension corresponding to the first target dimension data as the row dimension as an example, the first target dimension data to be calculated is determined to be the data in the second row of the weight matrix.

[0051] S2: Obtain first output vector data according to the first value and the first target dimension data.

[0052] In the embodiment of the present disclosure, after determining the first value and the first target dimension data to be involved in the calculation, only the part of the data can be loaded from the corresponding storage area without loading all the data, which can reduce the amount of data transmission.

[0053] In a possible embodiment, the first value is multiplied by the first target dimension data to obtain first output vector data. For example, if the first value is 1 and the first target dimension data is 1*5 row data, the first value is multiplied by the first target dimension data to obtain 1*5 first output vector data.

[0054] Step 130: Determine second target dimensional data based on the first output vector data, wherein the second target dimensional data and the first target dimensional data respectively belong to row dimensional data or column dimensional data in the weight matrix of the first network layer, and have no overlapping elements.

[0055] In a possible embodiment, determining the second target dimension data according to the first output vector data includes:

[0056] 1) Determine a second value of an element in the first output vector data that complies with a second loading strategy and determine that the second value corresponds to a second index value in the first output vector data.

[0057] For example, the second loading strategy is to filter out the N values ​​with the largest absolute values ​​of the elements in the first output vector data, such as N is 3, and the first output vector data is a 1*5 vector, sort the absolute values ​​of the elements in the first output vector data, filter out the three largest second values, and obtain the second index values ​​of the filtered out three second values, such as the second index values ​​are 1, 2 and 5 respectively, that is, the filtered out three second values ​​are the values ​​of the first, second and fifth elements in the first output vector data respectively.

[0058] 2) According to the second index value, determine the corresponding second target dimension data in the weight matrix.

[0059] In the disclosed embodiment, the second target dimension data represents the second data to be calculated in the weight matrix of the first network layer. The second target dimension data and the first target dimension data belong to data of different dimensions. If the first target dimension data is row dimension data, the second target dimension data corresponds to column dimension data, and there are no overlapping elements between the second target dimension data and the first target dimension data. In this way, repeated calculations can be avoided.

[0060] For example, the second index values ​​are 1, 2, and 5 respectively. Taking the column dimension data corresponding to the second target dimension data as an example, the weight matrix of the first network layer is a 4*5-dimensional matrix, then the data in the 1st, 2nd, and 5th columns are filtered out from the weight matrix. However, there are element data in the three filtered columns that overlap with the first target dimension data. Therefore, it is also necessary to remove the values ​​of the elements in the three filtered columns that overlap with the first target dimension data.

[0061] The present disclosure also provides several possible implementations:

[0062] In one embodiment, the corresponding second target dimension data in the weight matrix is ​​determined according to the second index value, including: according to the second index value, all values ​​of the corresponding second target dimension in the weight matrix are obtained, and the values ​​of all the values ​​of the second target dimension that overlap with the first target dimension data are set to 0 to obtain the second target dimension data.

[0063] In this implementation, taking the second index values ​​as 1, 2 and 5 respectively as examples, the data of the 1st, 2nd and 5th columns of the weight matrix can be loaded from the storage area of ​​the weight matrix, and then the values ​​of the elements in the acquired data of the 1st, 2nd and 5th columns that overlap with the first target dimension data are set to 0. In this way, the elements that have participated in the calculation before do not need to participate in the calculation again, thereby further reducing the amount of calculation.

[0064] In another embodiment, the second target dimension data corresponding to the weight matrix is ​​determined according to the second index value, including: according to the second index value and the first target dimension data, obtaining the dimension data of the second target dimension corresponding to the weight matrix that is not the first target dimension data as the second target dimension data.

[0065] In this implementation, when loading the filtered dimensional data of the weight matrix from the storage area of ​​the weight matrix, if it is an element overlapping with the first target dimensional data, no loading, reading and calculation are performed. Instead, the dimensional data of the second target dimension that is not the first target dimensional data is directly read from the storage area, and the read data is used as the second target dimensional data. In this way, not only the amount of calculation can be reduced, but also the amount of data transmission can be further reduced.

[0066] For example, taking the second index values ​​as 1, 2 and 5 respectively, and the first target dimension data is the data in the second row of the weight matrix, then from the storage area of ​​the weight matrix, only the data in the first column, second column and fifth column except the second row need to be loaded. In this implementation, when reading data, the transmission of part of the data can be further reduced, the amount of data transmission can be reduced, and the reasoning speed can be improved.

[0067] Step S140: Perform a second sparse processing on the input vector data and the second target dimension data to obtain second output vector data.

[0068] In the embodiment of the present disclosure, after the second target dimension data to be involved in the calculation is determined, the input vector data and the second target dimension data can be multiplied to obtain the second output vector data.

[0069] It should be noted that, since only part of the data in the weight matrix and all elements of the input vector data are screened out for calculation during the second sparse processing, in order to ensure compliance with the operation rules and to align the dimensions of the final second output vector data and the first output vector data, the index position of the second target dimension data in the weight matrix can be determined according to the first index value and the second index value, and then expanded to a matrix with the same dimensional size as the weight matrix according to the second target dimension data. The values ​​at the index positions corresponding to the weight matrix in the expanded matrix are the second target dimension data, and the values ​​at other index positions are set to 0 (that is, they are not actually involved in the calculation), so that the expanded matrix and the input vector data are multiplied to obtain the second output vector data.

[0070] In this way, only the screened second target dimension data in the weight matrix of the first network layer participates in the calculation, and other data in the weight matrix are not calculated parameters, thus realizing sparse calculation and reducing the amount of calculation.

[0071] Step S150: Obtain target output vector data according to the first output vector data and the second output vector data.

[0072] With respect to step S150, in a possible embodiment, the first output vector data and the second output vector data are added to obtain target output vector data.

[0073] In another possible embodiment, based on a preset function, the first output vector data and the second output vector data are fused to obtain target output vector data.

[0074] There is no limitation on the preset function, for example, it may be an addition operation including weight coefficients, etc. For another example, the first output vector data and the second output vector data are subjected to a certain function operation to obtain the final target output vector data.

[0075] Among them, the obtained target output vector data, that is, as the final output of the first network layer, can be transmitted to the next second network layer of the first network layer. The second network layer obtains the target output vector data and can continue to calculate based on the data processing method in the above-mentioned embodiment of the present disclosure to obtain the final output of the second network layer.

[0076] In addition, in the embodiment of the present disclosure, if the first network layer is the last network layer in the network model, the target output vector data is obtained and does not need to be transmitted to the next network layer. It can be output immediately to obtain the final output result of this reasoning of the network model.

[0077] In the disclosed embodiment, for the first network layer in the network model, when performing calculations based on the input vector data and the weight matrix of the first network layer, the dimensional data in the input vector and the weight matrix can be screened to select more important data to participate in the calculation. In this way, through multi-stage sparse processing, only part of the data needs to be loaded and calculated, which can reduce the amount of data loading and calculation, thereby speeding up the reasoning speed of the network model and reducing power consumption.

[0078] The data processing method according to the embodiment of the present disclosure is described below.

[0079] As mentioned above, in the embodiment of the present disclosure, based on the first loading strategy, the input vector data is screened to determine the elements to be calculated, and then the first target dimension data to be calculated in the weight matrix is ​​determined to achieve sparse processing of data loading and calculation. In the embodiment of the present disclosure, several possible embodiments are provided for the first loading strategy:

[0080] In a possible embodiment, M values ​​of the elements in the input vector data with the largest absolute values ​​are determined, where M is an integer greater than or equal to 1.

[0081] That is, in this embodiment, the value with the largest absolute value is screened out for loading and calculation. The value with the largest absolute value usually has a larger weight in the calculation. By screening with the largest absolute value, the more important data participating in the calculation can be determined, reducing the impact on the actual calculation results.

[0082] In a possible embodiment, the largest M values ​​among the values ​​of each element in the input vector data are determined.

[0083] In a possible embodiment, M values ​​having the largest reciprocals of the absolute values ​​of the values ​​of the elements in the input vector data are determined.

[0084] In a possible embodiment, the first loading strategy is determined based on a preset processing rule of the second network layer, and different first loading strategies have different screening methods for elements in the vector data.

[0085] In this embodiment, the first loading strategy of the first network layer in the network model can be dynamically determined. The first loading strategies of different first network layers can be different, and the specific first loading strategy is not limited in the embodiment of the present disclosure.

[0086] For example, the second network layer after the first network layer is an activation function operation, and negative values ​​in the activation function operation will be set to 0. Therefore, when calculating in the first network layer, it is more appropriate to use the first loading strategy of sorting by value and filtering out the largest value. The data that does not participate in the calculation has little effect on the final output result of the network model.

[0087] For example, the next second network layer of the first network layer is a reciprocal operation. Therefore, when calculating in the first network layer, the first loading strategy is to sort by the reciprocal of the absolute value and filter out the value with the largest reciprocal of the absolute value, which has little impact on the accuracy of the final output result.

[0088] In this way, in the embodiment of the present disclosure, a more appropriate first loading strategy with less impact on the reasoning result can be dynamically determined based on the preset processing rules of the first network layer corresponding to the next second network layer, thereby reducing the loss and impact on the accuracy and precision of the reasoning results while reducing the amount of data transmission and calculation.

[0089] Among them, for the value of M in the first loading strategy, the present disclosure also provides several possible determination methods:

[0090] The value of M is preset and fixed; or, the value of M is determined according to currently available resource information.

[0091] For example, the value of M may be preset and fixed based on actual conditions and experience, and there is no restriction on this.

[0092] For another example, currently available resource information may include computing resources, storage resources, communication bandwidth resources, etc. Currently available resource information is negatively correlated with the value of M. If currently available resource information is less, a smaller M may be determined, which may reduce the data involved in the calculation.

[0093] In the embodiment of the present disclosure, several possible embodiments are also provided for the second loading strategy:

[0094] In a possible embodiment, N values ​​with the largest absolute values ​​of the elements in the first output vector data are determined, where N is an integer greater than or equal to 1.

[0095] In a possible embodiment, the largest N values ​​among the values ​​of each element in the first output vector data are determined.

[0096] In a possible embodiment, N values ​​having the largest reciprocals of the absolute values ​​of the values ​​of the elements in the first output vector data are determined.

[0097] In a possible embodiment, the second loading strategy is determined based on a preset processing rule of the second network layer, and different second loading strategies have different screening methods for elements in the vector data.

[0098] Specifically, the second loading strategy is dynamically determined according to the preset processing rules of the second network layer, which is similar to the determination of the first loading strategy mentioned above and will not be described in detail here.

[0099] Preferably, the first loading strategy and the second loading strategy are not restricted in the disclosed embodiment, but in order to further improve the calculation accuracy, the principles of the data to be calculated screened out by the two-stage sparse processing are consistent. Therefore, the screening method in the first loading strategy and the second loading strategy is the same, but the values ​​of M and N do not need to remain the same.

[0100] Among them, for the value of N in the second loading strategy, the present disclosure also provides several possible determination methods:

[0101] 1) The value of N includes any of the following methods: the value of N is pre-set; the value of N is determined based on currently available resource information; the value of N is determined based on the median and mean corresponding to the values ​​of each element in the first output vector data.

[0102] For example, similar to the value of M, the value of N may also be preset and fixed, or determined according to currently available resource information. The currently available resource information and the value of N are also negatively correlated.

[0103] 2) The value of N is determined in the following manner: determine the median and the mean based on the values ​​of each element in the first output vector data; determine the difference between the median and the mean; and determine the value of N based on the difference, wherein the size of the difference is positively correlated with the size of the value of N.

[0104] For example, the value of N can be determined based on the median and mean corresponding to the values ​​of each element in the first output vector data. If the difference between the median and the mean is small, it means that the first output vector data is already relatively accurate, so a smaller N can be used to complete subsequent calculation compensation. Conversely, if the difference between the median and the mean is large, N can be increased to further improve the calculation accuracy.

[0105] One possible calculation method is: N = N0 (βsigmoid (α|median-mean|) + 1-β)

[0106] Among them, sigmoid is an S-shaped function with a value ranging from 0 to 1, N0 is the total dimension of the first output vector data, β and α are fixed sparse factors, generally limited to β<1, α>0, median is the median, and mean is the mean.

[0107] In this way, in the embodiment of the present disclosure, by dynamically determining the values ​​of M and N, the adaptability of multi-stage sparse computing can be further improved, the amount of data transmission and calculation can be reduced, and the reasoning performance of the network model can be improved.

[0108] In addition, in the embodiments of the present disclosure, the data processing method can be applied to a many-core system, which includes multiple processing cores, each of which includes a computing unit and a storage unit. The data processing method can be executed by the computing unit in the processing core. During the calculation process, the computing unit can obtain the required first target dimension data, second target dimension data and other data from the storage unit in the processing core where the computing unit is located or other processing cores. In this way, the amount of data transmission between the storage unit and the computing unit can be reduced, and the amount of calculation of the computing unit can be reduced, thereby improving the reasoning speed and performance of the many-core system.

[0109] The data processing method in the embodiment of the present disclosure is described below using a specific application scenario. Figure 2 , which is an example diagram of a data processing method in an embodiment of the present disclosure.

[0110] like Figure 2 As shown, for example, the input vector data of the first network layer in the network model is (0, 2, -4, 1), and the weight matrix of the first network layer is

[0111] In the data processing method in the embodiment of the present disclosure, the calculation process for the first network layer can be divided into two stages:

[0112] The first stage: Taking the first loading strategy as an example to determine the value with the largest absolute value of each element in the input vector data, the absolute values ​​of the four elements in the input vector data are sorted, and the first value with the largest absolute value is determined to be -4. The first value -4 corresponds to the first index value of the third in the input vector data. According to the first index value, the first target dimension data corresponding to the weight matrix of the first network layer is determined. Taking the first target dimension data as row dimension data as an example, the data of the third row dimension in the weight matrix is ​​used as the first target dimension data, that is, Figure 2 In the figure, the first target dimension data is (-1,-1,0,0,-1), the first value is -4, and the first value is multiplied by the first target dimension data to obtain the first output vector data, and the first output vector data is (4,4,0,0,4).

[0113] The second stage: Taking the second loading strategy as an example, the absolute values ​​of the values ​​of each element in the first output vector data are sorted according to the second loading strategy, and the three second values ​​with the largest absolute values ​​are determined, and the second index value in the first output vector data is determined, such as Figure 2 As shown, the determined second values ​​are 4, 4 and 4, which correspond to the second index values ​​​​in the first output vector data are the 1st, 2nd and 5th respectively.

[0114] Then, according to the second index value, the corresponding second target dimension data in the weight matrix is ​​determined. The second target dimension data and the first target dimension data belong to row dimension data or column dimension data respectively, and there are no overlapping elements, such as Figure 2 As shown, the data in the first column, the second column and the fifth column in the weight matrix are filtered out, and the data in the third row are not included, and the filtered data are used as the second target dimension data.

[0115] According to the input vector data and the second target dimension data, the second output vector data is obtained, such as Figure 2 As shown, the first output vector data is a 4-dimensional vector, and the determined second output vector data is 3*3 data. In order to comply with the operation rules of matrices and vectors, ensure that the dimensions of the final second output vector data are the same as those of the first output vector data, and ensure the accuracy of the calculation, only the determined second target dimension data can be retained in the weight matrix, and the values ​​of other positions are set to 0, that is, the matrix Multiplying the matrix and the input vector data will yield the second output vector data (4, 3, 0, 0, 4).

[0116] Thus, according to the first output vector data and the second output vector data, the target output vector data is obtained, such as Figure 2 As shown, taking the addition fusion method as an example, the first output vector data (4,4,0,0,4) and the second output vector data (4,3,0,0,4) are added to obtain the target output vector data (8,7,0,0,8).

[0117] Thus, in the disclosed embodiment, the calculation of the network layer in the network model is divided into a plurality of stages. In the first stage, the input vector data and the weight matrix are subjected to a first sparse processing, that is, a sparse calculation of the input dimension of the weight matrix is ​​performed, and only the first value screened out in the input vector data and the corresponding first target dimension data in the weight matrix are calculated to obtain the first output vector data. In the second stage, according to the first output vector data, a second sparse processing is performed on the output dimension of the weight matrix, and the input vector data and the second target dimension data in the weight matrix are calculated to obtain the second output vector data. Then, through the synthesis of the two stages, the final target output vector data is obtained, and the precision loss is compensated for each other, reducing the impact on accuracy. In addition, only part of the data needs to be loaded and calculated, which can reduce the amount of data transmission and calculation, thereby improving the reasoning speed and performance of the network model.

[0118] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

[0119] In addition, the present disclosure also provides a data processing device, an electronic device, a computer-readable storage medium, and a computer program product, all of which can be used to implement any data processing method provided by the present disclosure. The corresponding technical solutions and descriptions are referred to the corresponding records in the method part and will not be repeated here.

[0120] Figure 3 A block diagram of a data processing device provided in an embodiment of the present disclosure.

[0121] Reference Figure 3 , an embodiment of the present disclosure provides a data processing device, the data processing device comprising:

[0122] An acquisition module 31 is used to acquire input vector data of the first network layer in the network model;

[0123] A first processing module 32, configured to perform a first sparse processing on the input vector data and the first target dimension data to obtain first output vector data;

[0124] A determination module 33 is used to determine second target dimensional data according to the first output vector data, wherein the second target dimensional data and the first target dimensional data belong to row dimensional data or column dimensional data in the weight matrix of the first network layer respectively, and have no overlapping elements;

[0125] A second processing module 34, configured to perform a second sparse processing on the input vector data and the second target dimension data to obtain second output vector data;

[0126] The third processing module 35 is used to obtain target output vector data according to the first output vector data and the second output vector data.

[0127] In a possible embodiment, when performing the first sparse processing on the input vector data and the first target dimension data to obtain the first output vector data, the first processing module 32 is used to:

[0128] Determine, according to the input vector data and a first loading strategy, a first value of an element to be calculated in the input vector data and first target dimension data to be calculated in a weight matrix of the first network layer, wherein the first loading strategy is used to characterize a method for screening elements in the vector data;

[0129] According to the first value and the first target dimension data, first output vector data is obtained.

[0130] In a possible embodiment, when determining the first value of the element to be calculated in the input vector data and the first target dimension data to be calculated in the weight matrix of the first network layer according to the input vector data and the first loading strategy, the first processing module 32 is used to:

[0131] Determine a first value of an element in the input vector data that complies with a first loading strategy and a first index value in the input vector data corresponding to the first value;

[0132] According to the first index value, the first target dimension data corresponding to the weight matrix of the first network layer is determined, wherein the first value and the first target dimension data are the data to be calculated.

[0133] In a possible embodiment, when determining the second target dimension data according to the first output vector data, the determination module 33 is used to:

[0134] Determine a second value of an element in the first output vector data that complies with a second loading strategy and a second index value in the first output vector data corresponding to the second value;

[0135] According to the second index value, the corresponding second target dimension data in the weight matrix is ​​determined.

[0136] In a possible embodiment, the first loading strategy includes any one of the following:

[0137] Determine M values ​​with the largest absolute values ​​of the values ​​of each element in the input vector data;

[0138] Determine the largest M values ​​among the values ​​of each element in the input vector data;

[0139] Determine M values ​​whose reciprocals of the absolute values ​​of the values ​​of the elements in the input vector data are the largest;

[0140] Wherein, M is an integer greater than or equal to 1.

[0141] In a possible embodiment, the value of M is preset and fixed; or, the value of M is determined according to currently available resource information.

[0142] In a possible embodiment, the second loading strategy includes any one of the following:

[0143] Determine N values ​​with the largest absolute values ​​of the values ​​of each element in the first output vector data;

[0144] Determine the largest N values ​​among the values ​​of each element in the first output vector data;

[0145] Determine N values ​​whose reciprocals of the absolute values ​​of the values ​​of the elements in the first output vector data are the largest;

[0146] Wherein, N is an integer greater than or equal to 1.

[0147] In a possible embodiment, the value of N includes any of the following methods:

[0148] The value of N is preset and fixed;

[0149] The value of N is determined according to currently available resource information;

[0150] The value of N is determined based on the median and mean values ​​corresponding to the values ​​of each element in the first output vector data.

[0151] In a possible embodiment, the value of N is determined by:

[0152] Determine the median and the mean according to the value of each element in the first output vector data;

[0153] determining a difference between the median and the mean;

[0154] The value of N is determined according to the difference, wherein the size of the difference is positively correlated with the size of the value of N.

[0155] In a possible embodiment, when determining the corresponding second target dimension data in the weight matrix according to the second index value, the second processing module 34 is used to:

[0156] According to the second index value, all values ​​of the corresponding second target dimension in the weight matrix are obtained, and the values ​​of all the values ​​of the second target dimension that overlap with the first target dimension data are set to 0 to obtain the second target dimension data; or,

[0157] According to the second index value and the first target dimension data, dimension data other than the first target dimension data in the corresponding second target dimension in the weight matrix is ​​obtained as the second target dimension data.

[0158] In a possible embodiment, when obtaining the target output vector data according to the first output vector data and the second output vector data, the third processing module 35 is used to:

[0159] Adding the first output vector data and the second output vector data to obtain target output vector data; or,

[0160] Based on a preset function, the first output vector data and the second output vector data are fused to obtain target output vector data.

[0161] Figure 4 A block diagram of an electronic device provided in an embodiment of the present disclosure.

[0162] Reference Figure 4 An embodiment of the present disclosure provides an electronic device, which includes multiple processing cores 401 and an on-chip network 402, wherein the multiple processing cores 401 are connected to the on-chip network 402, and the on-chip network 402 is used to exchange data between the multiple processing cores and external data.

[0163] One or more instructions are stored in one or more processing cores 401 , and the one or more instructions are executed by one or more processing cores 401 , so that one or more processing cores 401 can execute the above-mentioned data processing method.

[0164] In some embodiments, the electronic device may be a brain-like chip, which can use vectorized computing and needs to load the weight information and other parameters of the neural network model through an external memory such as a double data rate (DDR) synchronous dynamic random access memory. Therefore, the batch processing used in the embodiments of the present disclosure has a higher computing efficiency.

[0165] The present disclosure also provides a computer-readable storage medium on which a computer program is stored, wherein the computer program implements the above-mentioned data processing method when executed by a processor or a processing core. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0166] An embodiment of the present disclosure also provides a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor or a processing core of an electronic device, the processor or the processing core in the electronic device executes the above-mentioned data processing method.

[0167] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium).

[0168] As is known to those of ordinary skill in the art, the term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable program instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, it is known to those of ordinary skill in the art that communication media typically contain computer-readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and may include any information delivery medium.

[0169] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0170] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0171] The computer program product described herein may be implemented in hardware, software, or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), etc.

[0172] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0173] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0174] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0175] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0176] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly noted, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims

1. A data processing method, characterized in that: include: Get the input vector data of the first network layer in the network model; Performing a first sparse processing on the input vector data and the first target dimension data to obtain first output vector data; Determine second target dimensional data according to the first output vector data, wherein the second target dimensional data and the first target dimensional data belong to row dimensional data or column dimensional data in the weight matrix of the first network layer respectively, and have no overlapping elements; Performing a second sparse processing on the input vector data and the second target dimension data to obtain second output vector data; Target output vector data is obtained according to the first output vector data and the second output vector data.

2. The method according to claim 1, characterized in that The performing a first sparse processing on the input vector data and the first target dimension data to obtain first output vector data includes: Determine, according to the input vector data and a first loading strategy, a first value of an element to be calculated in the input vector data and first target dimension data to be calculated in a weight matrix of the first network layer, wherein the first loading strategy is used to characterize a method for screening elements in the vector data; According to the first value and the first target dimension data, first output vector data is obtained.

3. The method according to claim 2, characterized in that The determining, according to the input vector data and the first loading strategy, a first value of an element to be calculated in the input vector data and first target dimension data to be calculated in the weight matrix of the first network layer includes: Determine a first value of an element in the input vector data that complies with a first loading strategy and a first index value in the input vector data corresponding to the first value; According to the first index value, the first target dimension data corresponding to the weight matrix of the first network layer is determined, wherein the first value and the first target dimension data are the data to be calculated.

4. The method according to claim 1, characterized in that: The step of determining second target dimension data according to the first output vector data includes: Determine a second value of an element in the first output vector data that complies with a second loading strategy and a second index value in the first output vector data corresponding to the second value; According to the second index value, the corresponding second target dimension data in the weight matrix is ​​determined.

5. The method according to any one of claims 2 or 3, characterized in that: The first loading strategy includes any one of the following: Determine M values ​​with the largest absolute values ​​of the values ​​of each element in the input vector data; Determine the largest M values ​​among the values ​​of each element in the input vector data; Determine M values ​​whose reciprocals of the absolute values ​​of the values ​​of the elements in the input vector data are the largest; Wherein, M is an integer greater than or equal to 1.

6. The method according to claim 5, characterized in that The value of M is preset and fixed; or, the value of M is determined according to currently available resource information.

7. The method according to claim 4, characterized in that The second loading strategy includes any one of the following: Determine N values ​​with the largest absolute values ​​of the values ​​of each element in the first output vector data; Determine the largest N values ​​among the values ​​of each element in the first output vector data; Determine N values ​​whose reciprocals of the absolute values ​​of the values ​​of the elements in the first output vector data are the largest; Wherein, N is an integer greater than or equal to 1.

8. The method according to claim 7, characterized in that The value of N includes any of the following methods: The value of N is preset and fixed; The value of N is determined according to currently available resource information; The value of N is determined based on the median and mean values ​​corresponding to the values ​​of each element in the first output vector data.

9. The method according to claim 7, characterized in that: The value of N is determined by: Determine the median and the mean according to the value of each element in the first output vector data; determining a difference between the median and the mean; The value of N is determined according to the difference, wherein the size of the difference is positively correlated with the size of the value of N.

10. The method according to claim 4, characterized in that The step of determining the corresponding second target dimension data in the weight matrix according to the second index value includes: According to the second index value, all values ​​of the corresponding second target dimension in the weight matrix are obtained, and the values ​​of all the values ​​of the second target dimension that overlap with the first target dimension data are set to 0 to obtain the second target dimension data; or, According to the second index value and the first target dimension data, dimension data other than the first target dimension data in the corresponding second target dimension in the weight matrix is ​​obtained as the second target dimension data.

11. The method according to claim 1, characterized in that: The step of obtaining target output vector data according to the first output vector data and the second output vector data comprises: Adding the first output vector data and the second output vector data to obtain target output vector data; or, Based on a preset function, the first output vector data and the second output vector data are fused to obtain target output vector data.

12. A data processing device, characterized in that: include: An acquisition module, used to acquire input vector data of the first network layer in the network model; A first processing module, configured to perform a first sparse processing on the input vector data and the first target dimension data to obtain first output vector data; A determination module, configured to determine second target dimensional data according to the first output vector data, wherein the second target dimensional data and the first target dimensional data respectively belong to row dimensional data or column dimensional data in the weight matrix of the first network layer, and have no overlapping elements; A second processing module, used for performing a second sparse processing on the input vector data and the second target dimension data to obtain second output vector data; The third processing module is used to obtain target output vector data according to the first output vector data and the second output vector data.

13. An electronic device, characterized in that: include: Multiple processing cores; as well as An on-chip network is configured to exchange data between the multiple processing cores and external data; wherein one or more instructions are stored in one or more of the processing cores, and one or more of the instructions are executed by one or more of the processing cores, so that one or more of the processing cores can execute the data processing method described in any one of claims 1-11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the data processing method according to any one of claims 1 to 11.

Images