Neural network training method involving sparse matrix multiplication using different mask blocks and computing device

Abstract

The present disclosure provides a neural network training method and a computing device, which relates to the technical field of artificial intelligence. The scheme includes: splitting a weight matrix into В weight blocks each having the same size and satisfying N:M sparsity constraint; and then by using В sparse mask sets defined for the В weight blocks respectively and В random number indices generated for the В weight blocks respectively, generating a mask matrix for implementing sparse matrix multiplication operation. There is no need to pre-train a dense weight matrix of huge-scale in advance nor to fine-tune a trained model again for the N:M sparsity constraint, which saves a lot of training time and improves training efficiency.

Description

SPECIFICATIONNEURAL NETWORK TRAINING METHOD INVOLVING SPARSE MATRIX MULTIPLICATION USING DIFFERENT MASK BLOCKS AND COMPUTING DEVICETECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, in particular to a neural network training method and a computing device.BACKGROUND

[0002] In the recent years, Deep Learning has reached significant breakthroughs in many practical problems, such as Computer Vision (e.g., object detection, segmentation and face identification), Natural Language Processing and Speech Recognition. In fact, neural networks, which are the main instrument of Deep Learning, have become ubiquitous. The neural networks are used to search information, to translate speech and text, to drive vehicles, to control the quality of products on manufactories, to do weather forecast, to improve quality of photographs, to recommend goods and to do many other everyday activities.

[0003] Although the applications of the neural networks are very diverse, they are typically based on one or several of the following foundation operations: Convolutions, Multi-Head Self Attention (MHSA), Fully-Connected Layers (e.g., Multi-layer Perceptron, MLP for short).

[0004] All of the above operations can be implemented using matrix multiplication. Thus, to accelerate model training, it is important to carry out the matrix multiplication as efficiently as possible. In light of this, an N:M sparsity method of matrix multiplication is proposed, where N and M are both positive integers, and M > N. For example, N:M may be 2:4 or 4:8. As shown in FIG. 1 , in each 4 (i.e., M = 4) sequential elements of left matrix row (or, possibly, right matrix column), 2 (i.e., N = 2) elements can be set to zero. Such assumption can theoretically accelerate the matrix multiplication by 2 (i.e., M / N) times. However, the key limitation of this method is that dense pre-trained model is needed, and this model is further needed to be fine-tuned for N:M sparsity constraint.

[0005] As further shown in FIG. 2, a transposable N:M sparse mask is proposed, which allows to utilize hardware feature of structured sparsity to accelerate model training. Also, a 2-approxi mation method for finding the optimal sparse mask is proposed, but it requires extra training time overhead. Moreover, the model still needs to be fine-tuned for the N:M sparsity constraint, so it is impossible to fully utilize the hardware feature to accelerate model training.SUMMARY

[0006] Embodiments of the present disclosure provide a neural network training method and a computing device, which does not need to pre-train a dense weight matrix of huge-scale in advance nor to fine-tune a trained model again for the N:M sparsity constraint, which saves a lot of training time and improves training efficiency

[0007] In first aspect, a neural network training method is provided. The method includes: splitting a weight matrix into B weight blocks, the B weight blocks having a same size and defining B sparse mask sets, each sparse mask set including D mask blocks satisfying N:M sparsity constraint, where N, M, B and D are all positive integers, N < M, B > 1 , and D > 1 ; generating B random number indices respectively corresponding to the B weight blocks, the B random number indices each having a random range of [1 ,D]; determining B mask blocks using the B random number indices and the B sparse mask sets, the B mask blocks corresponding to the B weight blocks; generating a mask matrix using the B mask blocks; and implementing sparse matrix multiplication operation using the mask matrix.

[0008] Based on the neural network training method provided in the first aspect, a weight matrix may be split into B weight blocks each having the same size and satisfying N:M sparsity constraint; then by using B sparse mask sets defined for the B weight blocks respectively and B random number indices generated for the B weight blocks respectively, a mask matrix for implementing sparse matrix multiplication operation is generated. There is no need to pre-train a dense weight matrix of huge-scale in advance nor to fine-tune a trained model again for the N:M sparsity constraint, which saves a lot of training time and improves training efficiency.

[0009] Here, satisfying the N:M sparsity constraint may include that all mask blocks in the B sparse mask sets satisfy the N:M sparsity constraint by rows and / or by columns, which is not limited by embodiments of the present disclosure. Optionally, the N:M sparsity constraint may be that all mask blocks satisfy the N:M sparsity constraint by rows and by columns, so that the mask matrix may be used during both the forward and backward neural network training processes to scale down the number of weights in the weight matrix that are actually used for training, thereby improving training efficiency.

[0010] Any one of the above mask blocks can be composed of elements that take the value of 0 or 1 , such as the binary numbers 0 or 1. In this way, during the neural network training process, elements in the mask matrix taking the value of 1 are actually used to select a portion of the weights, i.e., that portion of the weights that satisfy the N:M sparsity constraint, from the weight matrix for training, which improves the training efficiency by a factor of approximately M / N.

[0011] Specifically, the following schemes may be used to generate the B mask blocks for the B weight blocks and generate the mask matrix based on the B mask blocks.

[0012] In Schemel , i.e., an All randomization scheme, in one possible design, determining the B mask blocks using the B random number indices and the B sparse mask sets includes: randomly selecting B mask blocks for the B weight blocks from the B sparse mask sets respectively based on the B random number indices; and generating the mask matrix using the B mask blocks includes: generating the mask matrix using the B mask blocks selected from the B sparse mask sets, a position of an i-th mask block in the B mask blocks in the mask matrix coinciding with a position of an i-th weight block in the weight matrix, where i is a positive integer and 1 < i < B. In this way, all randomization is used to generate the B mask blocks for the B weight blocks respectively, and to generate the mask matrix, there is no need to pre-train a dense weight matrix of huge-scale in advance nor to fine-tune a training model again for the N:M sparsity constraint, which saves a lot of training time and improves training efficiency.

[0013] In Scheme 2, i.e., a Partial replacement scheme, in another possible design, determining the B mask blocks using the B random number indices and the B sparse mask sets includes: randomly selecting B mask blocks for the B weight blocks from the B sparse mask sets respectively based on the B random number indices; obtaining X additional mask blocks, the X additional mask blocks being mask blocks that give X weight blocks a highest score for highest numbers of times in one or more predefined durations, where X is an integer and 0 < X < B; and updating the B mask blocks by replacing X mask blocks of the B mask blocks selected from the B sparse mask sets with the X additional mask blocks; and generating the mask matrix using the B mask blocks includes: generating the mask matrix using the updated B mask blocks, a position of an i-th mask block in the updated B mask blocks in the mask matrix coinciding with a position of an i-th weight block in the weight matrix, where i is a positive integer and 1 < i < B. In this way, the X additional mask blocks that give the X weight blocks in the B weight blocks the highest score for the highest numbers of times in the one or more predefined durations can be used to replace the X mask blocks randomly generated for those X weight blocks, i.e. , some or all of the mask blocks with optimal long term performance in terms of training effects can be used to replace mask blocks with unknown training effects, to improve the overall quality of the B mask blocks that are actually used to generate the mask matrix, thereby improving the training effects.

[0014] Optionally, the i-th mask block in the mask matrix generated with the updated B mask blocks satisfies:Ai = Fi + (1-sign (Fi)) * Ri;

[0015] where Ai is an index of the i-th mask block in the mask matrix generated with the updated B mask blocks, Ri is an index of the i-th mask block of the B mask blocks selected from the B sparse mask sets, Fi is an index of a mask block corresponding to the i-th weight block in the X additional mask blocks, and sign (Fi) is an index sign function that satisfies: sign

[0016] where Fi < l or Fi > D indicates that a mask block corresponding to an i-th weight block in the Bweight blocks is not determined, and 1 < Fi < D indicates that the mask block corresponding to the i-th weight block in the B weight blocks is determined as a mask block with an index being a designated index Fi in an i-th sparse mask set.

[0017] That is to say, by means of indices of mask blocks that have been previously determined and stored (an initialization value of which less than 1 or greater than D, and an updated value of which greater than or equal to 1 and less than or equal to D), the randomly generated X mask blocks in the B mask blocks can be replaced with the X additional mask blocks that have been previously determined. For example, the " X additional mask blocks that have been determined" may be such mask blocks as described above that give the X weight blocks in the B weight blocks the highest score for the highest number of times, thereby improving the quality of the mask matrix and the training effects.

[0018] In Scheme 3, i.e., a Mergence scheme, in yet another possible design, determining the B mask blocks using the B random number indices and the B sparse mask sets includes: randomly selecting B-X mask blocks for B-X weight blocks from B-X sparse mask sets respectively based on B-X random number indices, where X is an integer and 0 < X < B; obtaining X additional mask blocks, the X additional mask blocks being mask blocks that give X weight blocks a highest score for highest numbers of times in one or more predefined durations; and updating the B mask blocks by merging the B-X mask blocks and the X additional mask blocks; and generating the mask matrix using the B mask blocks includes: generating the mask matrix using the updated B mask blocks, a position of an i-th mask block in the updated B mask blocks in the mask matrix coinciding with a position of an i-th weight block in the weight matrix, where i is a positive integer and 1 < i < B. That is to say, compared to the replacement scheme described above, if it is already known that there are the X additional mask blocks that give the X weight blocks in the B weight blocks the highest score for the highest number of times, the B mask blocks can be obtained by directly using and merging the X additional mask blocks and the B-X mask blocks randomly generated for the other B-X weight blocks, so as to generate the mask matrix, thereby reducing the workload of randomly generating and replacing some of the mask blocks and improving the training efficiency.

[0019] In one possible design, obtaining the X additional mask blocks includes: obtaining the X additional mask blocks from X sparse mask sets in the B sparse mask sets based on X designated indices.

[0020] Optionally, the method further includes: before obtaining the X additional mask blocks from the X sparse mask sets in the B sparse mask sets based on the X designated indices, determining X designated indices for X weight blocks in the B weight blocks respectively in the one or more predefined durations. That is, the X designated indices may be determined over a relatively long time, i.e., the X designated indices are indices of such mask blocks that perform well over a long time for the training effects of the X weight blocks in the B weight blocks, ensuring the quality of the mask matrix and the training effects.

[0021] Further, determining the X designated indices for the X weight blocks in the B weight blocks respectively in the one or more predefined durations includes: determining E weight blocks and E additional mask blocks in one predefined duration of the one or more predefined durations, the E additional mask blocks being mask blocks that give the E weight blocks a highest score for highest numbers of times in the one predefined duration, where E is an integer and 0 < E < X; determining the X weight blocks and the X additional mask blocks in the one or more predefined durations; and assigning the X designated indices of the X additional mask blocks to the X weight blocks respectively. That is to say, the X designated indices described above may be determined all in one predefined duration (E = X) or may be determined in batches over more predefined durations (E < X), e.g., the number (e.g., X / the total number of the predefined durations) or ratio of weight blocks and mask blocks to be determined for each predefined duration may be set, which are not limited by embodiments of the present disclosure.

[0022] Further again, determining the E weight blocks and the E additional mask blocks includes: setting all elements in a counter matrix of a size of B D and all elements in an index array of a size of B to 0, a counter in an i-th row and a j-th column in the count matrix being used for counting when a j-th mask block in the i-th sparse mask set gives the i-th weight block a highest score, an i-th element in the index array being used forstoring a designated index of a mask block corresponding to the i-th weight block, where j is a positive integer and 1 < j < D; once every certain number of training steps, determining a score the j-th mask block in the i-th sparse mask set gives the i-th weight block; finding an additional mask block that gives the highest score to the i-th weight block, and incrementing a value of a counter corresponding to the i-th weight block and the additional mask block by 1 ; and determining weight blocks and mask blocks corresponding to E counters with E highest values in the counter matrix as the E weight blocks and the E additional mask blocks, and storing designated indices of the E additional mask blocks.

[0023] Specifically, scores given to the i-th weight block by all mask blocks in the i-th sparse mask set corresponding to the i-th weight block may be calculated in the following two methods.

[0024] Optionally, the B weight blocks are all sub-weight matrices of a size of K « P, where K and P are positive integers, and K = M and / or P = M; a score the j-th mask block in the i-th sparse mask set gives the i-th weight block satisfies:Si,j = Z1*P|wi n| * m^n;

[0025] where Si j is the score the j-th mask block in the i-th sparse mask set gives the i-th weight block, wi nis an n-th weight value in the i-th weight block, and mtj nis an n-th mask value of the j-th mask block in the i-th sparse mask set, where n is a positive integer and 1 s n s K*P.

[0026] Alternatively, the B weight blocks are all sub-weight matrices of a size of K x p, where K and P are positive integers, and K = M and / or P = M; and the B sparse mask sets are all first sparse mask sets. Determining the score the j-th mask block in the i-th sparse mask set gives the i-th weight block includes: generating a reshaped matrix using the B weight blocks, the reshaped matrix having a size of B x (K * P) or (K * P) x B, and an i-th row or an i-th column of the reshaped matrix including absolute values of all weights of the i-th weight block in the B weight blocks; generating an all-mask matrix using a first sparse mask set, the all-mask matrix having a size of (K * P) x D or D x (K * P), a j-th column or a j-th row of the all-mask matrix including all mask values of a j-th mask block in the first sparse mask set, and positions of all mask values of the j-th mask block in the first sparse mask set in the all-mask matrix coinciding with positions of the absolute values of all weights of the i-th weight block in the reshaped matrix respectively; and obtaining a score matrix by performing matrix multiplication operation on the reshaped matrix and the all-mask matrix, the score matrix having a size of B x D or D x B, an i-th row or an i-th column in the score matrix storing D scores, and a j-th score being a matrix product of absolute values all weights in the i-th weight block and all mask values in the j-th mask block.

[0027] It can be seen that since any one element in a mask block takes the value of 0 or 1 , the score given to the i-th weight block by all mask blocks in the i-th sparse mask set corresponding to the i-th weight block is the sum of absolute values of weights selected from the i-th weight block by using elements taking the value of 1 in all mask blocks in the i-th sparse mask set, and the higher that score is, the better the sparsity training is.

[0028] In second aspect, a computing device is provided which includes a memory and at least one processor coupled to the memory. The memory is configured to store computer instructions that, when executed by the at least one processor, cause the computing device to: split a weight matrix into B weight blocks, the B weight blocks having a same size and defining B sparse mask sets, each sparse mask set including D mask blocks satisfying N:M sparsity constraint, where N, M, B and D are all positive integers, N < M, B > 1 , and D > 1 ; generate B random number indices respectively corresponding to the B weight blocks, the B random number indices each having a random range of [1 ,D]; determine B mask blocks using the B random number indices and the B sparse mask sets, the B mask blocks corresponding to the B weight blocks; generate a mask matrix using the B mask blocks; and implementing sparse matrix multiplication operation using the mask matrix.

[0029] In one possible design, the computer instructions, when executed by the at least one processor, cause the computing device to: randomly select B mask blocks for the B weight blocks from the B sparse mask sets respectively based on the B random number indices; and generate the mask matrix using the B maskblocks selected from the B sparse mask sets, a position of an i-th mask block in the mask matrix coinciding with a position of an i-th weight block of the B mask blocks in the weight matrix, where i is a positive integer and 1 < i < B.

[0030] In another possible design, the computer instructions, when executed by the at least one processor, cause the computing device to: randomly select B mask blocks for the B weight blocks from the B sparse mask sets respectively based on the B random number indices; obtain X additional mask blocks, the X additional mask blocks being mask blocks that give X weight blocks in the B weight blocks a highest score for highest numbers of times in one or more predefined durations, where X is an integer and 0 S X S B; update the B mask blocks by replacing X mask blocks of the B mask blocks selected from the B sparse mask sets with the X additional mask blocks; and generate the mask matrix using the updated B mask blocks, a position of an i-th mask block in the updated B mask blocks in the mask matrix coinciding with a position of an i-th weight block in the weight matrix, where i is a positive integer and 1 < i < B.

[0031] Optionally, the i-th mask block in the mask matrix generated with the updated B mask blocks satisfies:Ai = Fi + (1 - sign(Fi)) * Ri;

[0032] where Ai is an index of the i-th mask block in the mask matrix generated with the updated B mask blocks, Ri is an index of the i-th mask block of the B mask blocks selected from the B sparse mask sets, Fi is an index of a mask block corresponding to the i-th weight block in the X additional mask blocks, and sign (Fi) is an index sign function that satisfies:

[0033] where Fi < 1 or Fi > D indicates that a mask block corresponding to the i-th weight block in the B weight blocks is not determined, and 1 < Fi < D indicates that the mask block corresponding to the i-th weight block in the B weight blocks is determined as a mask block with a designated index Fi in an i-th sparse mask set.

[0034] In yet another possible design, the computer instructions, when executed by the at least one processor, cause the computing device to: randomly select B-X mask blocks for B-X weight blocks from B-X sparse mask sets respectively based on B-X random number indices, where X is an integer and 0 < X < B; obtain X additional mask blocks, the X additional mask blocks being mask blocks that give X weight blocks in the B weight blocks a highest score for highest numbers of times in one or more predefined durations; update the B mask blocks by merging the B-X mask blocks and the X additional mask blocks; and generate the mask matrix using the updated B mask blocks, a position of an i-th mask block in the updated B mask blocks in the mask matrix coinciding with a position of an i-th weight block in the weight matrix, where i is a positive integer and 1 < i < B.

[0035] In one possible design, the computer instructions, when executed by the at least one processor, cause the computing device to obtain the X additional mask blocks from X sparse mask sets in the B sparse mask sets based on X designated indices.

[0036] Optionally, the computer instructions, when executed by the at least one processor, cause the computing device to: before obtaining the X additional mask blocks from the X sparse mask sets in the B sparse mask sets based on the X designated indices, determine X designated indices for X weight blocks in the B weight blocks respectively in the one or more predefined durations.

[0037] Further, the computer instructions, when executed by the at least one processor, cause the computing device to: determine E weight blocks and E additional mask blocks in one predefined duration of the one or more predefined durations, the E additional mask blocks being mask blocks that give the E weight blocks a highest score for highest numbers of times in the one predefined duration, where E is an integer and 0 s E < X; determine the X weight blocks and the X additional mask blocks in the one or more predefined durations; and assign the X designated indices of the X additional mask blocks to the X weight blocksrespectively.

[0038] Further again, the computer instructions, when executed by the at least one processor, cause the computing device to: set all elements in a counter matrix of a size of B * D and all elements in an index array of a size of B to 0, a counter in an i-th row and a j-th column in the counter matrix being used for counting when a j-th mask block in the i-th sparse mask set gives the i-th weight block a highest score, an i-th element in the index array being used for storing a designated index of a mask block corresponding to the i-th weight block, where j is a positive integer and 1 < j < D; once every certain number of training steps, determining a score the j-th mask block in the i-th sparse mask set gives the i-th weight block; find an additional mask block that gives the highest score to the i-th weight block, and incrementing a value of a counter corresponding to the i-th weight block and the additional mask block by 1 ; and determine weight blocks and mask blocks corresponding to E counters with E highest values in the counter matrix as the E weight blocks and the E additional mask blocks, and store designated indices of the E additional mask blocks.

[0039] Optionally, the B weight blocks are all sub-weight matrices of a size of K * P, where K and P are positive integers, and K = M and / or P = M; a score the j-th mask block in the i-th sparse mask set gives the i-th weight block satisfies:Sy = Zi*P|w,.n| * miJn;

[0040] where Sjis the score the j-th mask block in the i-th sparse mask set gives the i-th weight block, winis an n-th weight value in the i-th weight block, and miJinis an n-th mask value of the j-th mask block in the i-th sparse mask set, where n is a positive integer and 1 < n < K*P.

[0041] Optionally, the B weight blocks are all sub-weight matrices of a size of K * P, where K and P are positive integers, K = M and / or P = M. the B sparse mask sets are all first sparse mask sets. The computer instructions, when executed by the at least one processor, cause the computing device to: generate a reshaped matrix using the B weight blocks, the reshaped matrix having a size of B « (K * P) or (K * P) x B, and an i-th row or an i-th column of the reshaped matrix including absolute values of all weights of the i-th weight block in the B weight blocks; generate an all-mask matrix using a first sparse mask set, the all-mask matrix having a size of (K ‘ P) x D or D « (K * P), a j-th column or a j-th row of the all-mask matrix including all mask values of a j-th mask block in the first sparse mask set, and positions of all mask values of the j-th mask block in the first sparse mask set in the all-mask matrix coinciding with positions of the absolute values of all weights of the i-th weight block in the reshaped matrix respectively; and obtain a score matrix by performing matrix multiplication operation on the reshaped matrix and the all-mask matrix, the score matrix having a size of B x D or D x B, an i-th row or an i-th column in the score matrix storing D scores, and a j-th score being a matrix product of the absolute values of all weights in the i-th weight block and all mask values in the j-th mask block.

[0042] In a third aspect, a computer-readable storage medium is provided, which has stored thereon computer instructions that, when executed by a computer, cause the computer to implement the method provided in any of the implementations in the first aspect.

[0043] In a fourth aspect, a computer program product is provided, which contains instructions that, when executed by a computer, cause the computer to implement the method provided in any of the implementations in the first aspect.

[0044] In a fifth aspect, a chip system is provided. The chip system includes a processing circuit and a storage medium.

[0045] The storage medium has stored thereon computer instructions that, when executed by the processing circuit, cause the chip system to implement the method provided in any of the implementations in the first aspect.BRIEF DESCRIPTION OF THE DRAWINGS

[0046] FIG. 1 is a schematic diagram showing a 2:4 sparsity method for matrix multiplication;

[0047] FIG. 2 is a schematic diagram showing transposable N:M sparsity masks;

[0048] FIG. 3 is a schematic diagram showing a structure of a computing device, in accordance with embodimentsof the present disclosure;

[0049] FIG. 4 is a schematic flow chart showing a neural network training method, in accordance with embodiments of the present disclosure;

[0050] FIG. 5 is a schematic flow chart showing an all randomization scheme, in accordance with embodiments of the present disclosure;

[0051] FIG. 6 shows an example of an all randomization scheme, in accordance with embodiments of the present disclosure;

[0052] FIG. 7 is a schematic flow chart showing a partial replacement scheme, in accordance with embodiments of the present disclosure;

[0053] FIG. 8 shows an example of a partial replacement scheme, in accordance with embodiments of the present disclosure;

[0054] FIG. 9 is a schematic flow chart showing a mergence scheme, in accordance with embodiments of the present disclosure;

[0055] FIG. 10 shows an example of a mergence scheme, in accordance with embodiments of the present disclosure;

[0056] FIG. 11 is a schematic flow chart showing a method of determining a designated index for a weight block, in accordance with embodiments of the present disclosure;

[0057] FIG. 12 shows an example of a counter matrix and a mask block index array, in accordance with embodiments of the present disclosure;

[0058] FIG. 13 shows an example of giving B weight blocks by D mask blocks in the first sparse mask set, in accordance with embodiments of the present disclosure; and

[0059] FIG. 14 shows an example of determining E designated indices for E weight blocks respectively, in accordance with embodiments of the present disclosure.DETAILED DESCRIPTION

[0060] Implementation of embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings of the specification.

[0061] FIG. 3 is a schematic diagram showing a structure of a computing device provided in embodiments of the present disclosure. The computing device can be an electronic device with computing power, such as a server, a supercomputer, a mainframe computer, a network device, a terminal device, a personal computer (PC), or a chip or system-on-chip (SoC) in the above-mentioned device. As shown in FIG. 3, the computing device 300 includes a first processor 301 , a second processor 302, and a communication line 303.

[0062] The first processor 301 may be a general-purpose processor, such as an advanced RISC machine (ARM) processor or X86 processor, which is primarily used to run process to control software. The software may include a series of instructions that can call the second processor 302 to accomplish complex operations (e.g., matrix multiplications). The second processor 302 may be a specialized processor, such as a graphics processing unit (GPU) or tensor processing unit (TPU) for dedicated to neural network training. The communication line 303 may be a bus, or an electrically coupled line that enables communication between the first processor 301 and the second processor 302.

[0063] Here, the first processor 301 can perform sparsity computation scheme provided by embodiments of the present disclosure to find mask blocks that satisfy N.M sparsity constraint by rows and / or by columns, and provide the mask blocks to the second processor 302 via the communication line 303; and the second processor 302 can generate a mask matrix based on the mask blocks provided by the first processor 301 , and perform sparse mask training using the generated mask matrix to improve training efficiency.

[0064] In some embodiments, the first processor 301 may also be referred to as a host or controller, and the second processor 302 may also be referred to as an accelerator or hardware accelerator, which are not limited by embodiments of the present disclosure.

[0065] Optionally, the computing device 300 may further include one or more of: a memory 304, a communicationinterface 305, or an input / output interface 306, in which the first processor 301 , the second processor 302, the memory 304, the communication interface 305, and the input / output interface 306 may be in communication with one another via the communication line 303.

[0066] The communication interface 305 is used for the computing device 300 to communicate with another device or communication network. The another communication network may be an Ethernet network, a radio access network (RAN), a wireless local area network (WLAN), or the like. The communication interface 305 may be any device, part, component, or sub-system that enables communication, such as a module, a circuit or a transceiver.

[0067] The communication line 303 is used to transmit information between various components included in the computing device 300.

[0068] The memory 304 is used to store instructions and data, in which the instructions may be a computer program and the data may include input data, output data, and intermediate data.

[0069] The memory 304 may be a read-only memory (ROM) or a static storage device of another type that can store static information and / or instructions; alternatively, the memory 304 may be a random access memory (RAM) or a dynamic storage device of another type that can store information and / or instructions; alternatively, the memory 304 may be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or another CD storage, an optical storage (including compact disc, laser disc, compact disc, digital versatile disc, and Blu-ray disc), a magnetic disc storage media or another magnetic storage device, or the like, which are not limited.

[0070] It will be noted that the memory 304 may exist independent of the first processor 301 and / or second processor 302, or may be integrated with the first processor 301 and / or second processor 302. The memory 304 may be located within the computing device 300 or outside of the computing device 300, which is not limited.

[0071] In one example, the first processor 301 and / or the second processor 302 may each include one or more CPUs, such as CPU0 and CPU1 in FIG. 3.

[0072] As an optional implementation, the computing device 300 may include multiple first processors 301 and / or multiple second processors 302, which are not limited by embodiments of the present disclosure.

[0073] The input / output interface 306 is used to connect various input / output devices such as a keyboard, a mouse, a microphone, a display, and a speaker, which can receive commands input by a user, and / or, output calculation results.

[0074] It will be noted that the computing device 300 may be a desktop computer, a portable computer, a network server, a cell phone, a tablet computer, a wireless terminal, an embedded device, a system-on-chip, or a device having a similar structure as in FIG. 3.

[0075] In addition, the composition and structure shown in FIG. 3 do not constitute a limitation of the computing device 300, and the computing device 300 may include more or fewer components than shown, or combine certain components, or have a different arrangement of components, in addition to the components shown in FIG. 3.

[0076] Furthermore, actions, terms, and the like involved in individual embodiments of the present disclosure may be cross-referenced without limitation. Name of a message, name of parameter in the message, or the like, for an interaction between various devices in embodiments of the present disclosure is only an example, and other names may be used in specific implementations, which is not limited by embodiments of the present disclosure.

[0077] A neural network training method provided by embodiments of the present disclosure is described in detail below in conjunction with FIGS. 4 to 14.

[0078] FIG. 4 is a schematic flow chart showing a neural network training method provided in embodiments of the present disclosure. As shown in FIG. 4, the method includes the following steps S401 to S405.

[0079] In S401 , a weight matrix is split into B weight blocks, the B weight blocks having the same size and defining B sparse mask sets, each sparse mask set including D mask blocks satisfying N:M sparsity constraint, where N, M, B and D are all positive integers, N < M, B > 1 , and D > 1.

[0080] In S402, B random number indices respectively corresponding to the B weight blocks are generated, the B random number indices each having a random range of [1,D],

[0081] In S403, B mask blocks are determined using the B random number indices and the B sparse mask sets, the B mask blocks corresponding to the B weight blocks.

[0082] In S404, a mask matrix is generated using the B mask blocks.

[0083] In S405, sparse matrix multiplication operation is implemented using the mask matrix.

[0084] Based on the neural network training method provided by embodiments of the present disclosure, a weight matrix may be split into B weight blocks each having the same size and satisfying N:M sparsity constraint; then by using B sparse mask sets defined for the B weight blocks respectively and B random number indices generated for the B weight blocks respectively, a mask matrix for implementing sparse matrix multiplication operation is generated. There is no need to pre-train a dense weight matrix of huge-scale in advance nor to fine-tune a trained model again for the N:M sparsity constraint, which saves a lot of training time and improves training efficiency.

[0085] Here, satisfying the N:M sparsity constraint may include that all mask blocks in the B sparse mask sets satisfy the N:M sparsity constraint by rows and / or by columns. Optionally, the N:M sparsity constraint may be that all mask blocks satisfy the N:M sparsity constraint by rows and by columns, so that the mask matrix may be used during both the forward and backward neural network training processes to scale down the number of weights in the weight matrix that are actually used for training, thereby improving training efficiency.

[0086] Any one of the above mask blocks can be composed of elements that take the value of 0 or 1 , such as the binary numbers 0 or 1. In this way, during the neural network training process, elements in the mask matrix taking the value of 1 are actually used to select a portion of the weights, i.e., that portion of the weights that satisfy the N:M sparsity constraint, from the weight matrix for training, which improves the training efficiency by a factor of approximately M / N.

[0087] Specifically, the following schemes may be used to generate the B mask blocks for the B weight blocks and generate the mask matrix based on the B mask blocks.

[0088] Schemel (All randomization scheme)

[0089] In one possible design, as shown in FIG. 5, S403, in which the B mask blocks are determined using the B random number indices and the B sparse mask sets, includes:

[0090] S4031 , randomly selecting B mask blocks for the B weight blocks from the B sparse mask sets respectively based on the B random number indices.

[0091] Here, the B random number indices can be generated by a random function that has a random range of [1 , D].

[0092] Accordingly, S404, in which the mask matrix is generated using the B mask blocks, includes:

[0093] S4041 , generating the mask matrix using the B mask blocks selected from the B sparse mask sets, a position of an i-th mask block in the B mask blocks in the mask matrix coinciding with a position of an i-th weight block in the weight matrix, where i is a positive integer and 1 < i < B.

[0094] The "coinciding in position" herein means that in a case of performing a sparse matrix multiplication operation, all elements in the i-th weight block in the weight matrix can be multiplied with exactly all elements in the i-th mask block in the mask matrix without adjusting the position and order to complete the operation process.

[0095] As an example, as shown in (a) of FIG. 6, the weight matrix is assumed to be split into 8 weight blocks numbered in order from 1 to 8. Then, as shown in (b) of FIG. 6, using all randomization, 8 random number indices of mask blocks each satisfying the N:M sparsity constraint can be generated for these 8 weight blocks respectively, and the generated random number indices are shown as italicized numbers in FIG. 6. After that, 8 mask blocks can be selected from corresponding 8 sparse mask sets respectively, based on these 8 random number indices, and the mask matrix can be generated using the selected 8 mask blocks, as shown in (c) of FIG. 6. Here, positions of the selected 8 mask blocks in the mask matrix coincide with positions of the 8 weight blocks in the weight matrix, respectively.

[0096] It will be noted that for weight blocks and mask blocks having a defined size (the mask block having the size corresponding to the size of the weight block), the maximum number of types of mask blocks that are capable ofsatisfying the N:M sparsity constraint is also determined. For example, assuming that the weight block in FIG. 6 has a size of 4 x 4 and the sparsity constraint of N:M = 2:4 is satisfied both by rows and by columns, there are a total of 90 types of mask blocks corresponding to indices 1 to 90, respectively; accordingly, the generated mask block indices may have the random range of [1 , D], where D is a positive integer and D = 90.

[0097] Further, if it is known that the training effects of some mask blocks perform well for a long time, a range of values for the random number indices may be narrowed to a subset composed of the indices of those mask blocks. Successive indices for these mask blocks may also be re-established for ease of operation, and embodiments of the present disclosure are not limited.

[0098] In this way, all randomization is used to generate the B mask blocks for the B weight blocks respectively, and to generate the mask matrix, there is no need to pre-train a dense weight matrix of huge-scale in advance nor to fine-tune a training model again for the N:M sparsity constraint, which saves a lot of training time and improves training efficiency.

[0099] Scheme 2 (Partial replacement scheme)

[0100] In another possible design, as shown in FIG. 7, S403, in which the B mask blocks are determined using the B random number indices and the B sparse mask sets, includes:

[0101] S4032, randomly selecting B mask blocks for the B weight blocks from the B sparse mask sets respectively based on the B random number indices;

[0102] S4033, obtaining X additional mask blocks, the X additional mask blocks being mask blocks that give X weight blocks a highest score for highest numbers of times in one or more predefined durations, where X is an integer and 0 < X < B; and

[0103] S4034, updating the B mask blocks by replacing X mask blocks of the B mask blocks selected from the B sparse mask sets with the X additional mask blocks.

[0104] Accordingly, S404, in which the mask matrix is generated using the B mask blocks, includes:

[0105] S4042, generating the mask matrix using the updated B mask blocks, a position of an i-th mask block in the updated B mask blocks in the mask matrix coinciding with a position of an i-th weight block in the weight matrix, where i is a positive integer and 1 < i < B.

[0106] In this way, the X additional mask blocks with relatively good training effects can be used to replace the X mask blocks randomly generated (training effects unknown due to random generation) for those X weight blocks, i.e. , some or all of the mask blocks with optimal long term performance in terms of training effects can be used to replace mask blocks with unknown training effects, to improve the overall quality of the B mask blocks that are actually used to generate the mask matrix, thereby improving the training effects.

[0107] Optionally, the i-th mask block in the mask matrix generated with the updated B mask blocks satisfies:Ai = Fi + (1-sign (Fi)) * Ri;

[0108] where Ai is an index of the i-th mask block in the mask matrix generated with the updated B mask blocks, Ri is an index of the i-th mask block of the B mask blocks selected from the B sparse mask sets, Fi is an index of a mask block corresponding to the i-th weight block in the X additional mask blocks, and sign (Fi) is an index sign function that satisfies:

[0109] where Fi < l or Fi > D indicates that a mask block corresponding to an i-th weight block in the B weight blocks is not determined, and 1 < Fi < D indicates that the mask block corresponding to the i-th weight block in the B weight blocks is determined as a mask block with an index being a designated index Fi in an i-th sparse mask set.

[0110] That is to say, by means of indices of mask blocks that have been previously determined and stored (an initialization value of which less than 1 or greater than D, and an updated value of which greater than or equal to 1 and less than or equal to D), the randomly generated X mask blocks in the B mask blocks can be replaced with the Xadditional mask blocks that have been previously determined. For example, the " X additional mask blocks that have been determined" may be such mask blocks as described above that give the X weight blocks in the B weight blocks the highest score for the highest number of times, thereby improving the quality of the mask matrix and the training effects.

[0111] As an example, in conjunction with FIG. 6 and as shown in (a) and (b) of FIG. 8, 8 random number indices are randomly generated for 8 weight blocks respectively, in which random number indices of mask blocks corresponding to the 1-st and 6-th weight blocks are 85 and 59 respectively, and it is known that for the 1-st and 6-th weight blocks, the training effects of mask blocks with designated indices of 48 and 75 respectively, are stable and efficient in the long term, i.e., B = 8 and X = 2; and then, as shown in (c) of FIG. 8, in the process of generating a mask matrix for these 8 weight blocks, the mask blocks with indices of 48 and 75 are used to replace the mask blocks with random number indices of 85 and 59 respectively. For easy distinction, the indices of the 1-st and 6-th weight blocks, and the random number indices and the designated indices of the corresponding mask blocks are shown in bold in FIG. 8.

[0112] Scheme 3 (Mergence scheme)

[0113] In yet another possible design, as shown in FIG. 9, S403, in which the B mask blocks are determined using the B random number indices and the B sparse mask sets, includes:

[0114] S4035, randomly selecting B-X mask blocks for B-X weight blocks from B-X sparse mask sets respectively based on B-X random number indices, where X is an integer and 0 < X < B;

[0115] S4036, obtaining X additional mask blocks, the X additional mask blocks being mask blocks that giveX weight blocks a highest score for highest numbers of times in one or more predefined durations; and

[0116] S4037, updating the B mask blocks by merging the B-X mask blocks and the X additional mask blocks.

[0117] Accordingly, S404, in which the mask matrix is generated using the B mask blocks, includes:

[0118] S4043, generating the mask matrix using the updated B mask blocks, a position of an i-th mask block in the updated B mask blocks in the mask matrix coinciding with a position of an i-th weight block in the weight matrix, where i is a positive integer and 1 < i < B.

[0119] As an example, in conjunction with FIG. 8 and as shown in (a) and (b) of FIG. 10, since it is already known that for the 1-st and 6-th weight blocks, the training effects of the mask blocks with designated indices of 48 and 75 respectively, are stable and efficient in the long term, 6 random number indices may be generated for only the other 6 weight blocks in these 8 weight blocks. In this way, as shown in (c) of FIG. 10, a mask matrix is generated using the mask blocks with designated indices of 48 and 75 and the other mask blocks corresponding to the 6 random number indices. For easy distinction, the indices of the 1-st and 6-th weight blocks and the designated indices of the corresponding mask blocks are shown in bold in FIG. 10.

[0120] It will be noted that a variable used to store an index of a mask block may be set with an initial value, which may be 0 (as shown in (b) of FIG. 10) or other values, such as 91 or 95, as long as the initial value is not within the random range of [1,D] of the index of the mask block, which is not limited by embodiments of the present disclosure.

[0121] That is to say, compared to the partial replacement scheme described above, if it is already known that there are the X additional mask blocks that give the X weight blocks in the B weight blocks the highest score for the highest number of times, the B mask blocks can be obtained by directly using and merging the X additional mask blocks and the B-X mask blocks randomly generated for the other B-X weight blocks, so as to generate the mask matrix. There is no need to generate random number indices repeatedly for these X weight blocks, thus reducing the workload of randomly generating and replacing some of the mask blocks and improving the training efficiency.

[0122] In one possible design, obtaining the X additional mask blocks in S4033 and S4036 above, includes:

[0123] obtaining the X additional mask blocks from X sparse mask sets in the B sparse mask sets based onX designated indices.

[0124] Optionally, the method further includes: before obtaining the X additional mask blocks from the X sparse mask sets in the B sparse mask sets based on the X designated indices,

[0125] determining X designated indices for X weight blocks in the B weight blocks respectively in the one or more predefined durations.

[0126] Here, a predefined duration may be set according to actual requirements, such as a period of time, one or more training cycles (epochs), or one or more training steps, which is not limited by embodiments of the present disclosure. That is, the X designated indices may be determined over a relatively long time, i.e. , the X designated indices are indices of such mask blocks that perform well over a long time for the training effects of the X weight blocks in the B weight blocks, ensuring the quality of the mask matrix and the training effects.

[0127] Further, as shown in FIG. 11 , determining the X designated indices for the X weight blocks in the B weight blocks respectively in the one or more predefined durations includes the following steps S1101 to S1103.

[0128] In S1101, E weight blocks and E additional mask blocks are determined in one predefined duration of the one or more predefined durations, the E additional mask blocks being mask blocks that give the E weight blocks a highest score for highest numbers of times in the one predefined duration, where E is an integer and 0< E < X.

[0129] Further again, S1101 described above, in which the E weight blocks and the E additional mask blocks are determined, includes the following Steps 1 to 4.

[0130] In Step 1 , all elements in a counter matrix of a size of B x D and all elements in an index array of a size of B are set to 0. As shown in (a) of FIG. 12, a counter in an i-th row and a j-th column in the count matrix is used for counting when a j-th mask block in the i-th sparse mask set gives the i-th weight block a highest score; and as shown in in (b) of FIG. 12, an i-th element in the index array is used for storing a designated index of a mask block corresponding to the i-th weight block, where ] is a positive integer and 1 < j < D.

[0131] In Step 2, once every certain number of training steps, a score the j-th mask block in the i-th sparse mask set gives the i-th weight block is determined.

[0132] Specifically, scores given to the i-th weight block by all mask blocks in the i-th sparse mask set corresponding to the i-th weight block may be calculated in the following two methods.

[0133] In method 1, the B weight blocks are all sub-weight matrices of a size of K * P, where K and P are positive integers, and K = M and / or P = M; a score the j-th mask block in the i-th sparse mask set gives the i-th weight block satisfies:

[0134] where S,., is the score the j-th mask block in the i-th sparse mask set gives the i-th weight block, wi nis an n-th weight value in the i-th weight block, and m,j nis an n-th mask value of the j-th mask block in the i-th sparse mask set, where n is a positive integer and 1 < n < K*P.

[0135] In method 2, the B weight blocks are assumed to be all sub-weight matrices of a size of K * P, where K and P are positive integers, and K = M and / or P = M; and the B sparse mask sets are all first sparse mask sets. As shown in FIG. 13, Step 2 described above, in which determining the score the j-th mask block in the i-th sparse mask set gives the i-th weight block, includes:

[0136] Step(a), generating a reshaped matrix using the B weight blocks, the reshaped matrix having a size of B x (K * P) or (K * P) x B, and an i-th row or an i-th column of the reshaped matrix including absolute values of all weights of the i-th weight block in the B weight blocks;

[0137] Step(b), generating an all-mask matrix using a first sparse mask set, the all-mask matrix having a size of (K * P) x D or D x (K * P), a j-th column or a j-th row of the all-mask matrix including all mask values of a j-th mask block in the first sparse mask set, and positions of all mask values of the j-th mask block in the first sparse mask set in the all-mask matrix coinciding with positions of the absolute values of all weights of the i-th weight block in the reshaped matrix respectively; and

[0138] Step(c), obtaining a score matrix by performing matrix multiplication operation on the reshaped matrix and the all-mask matrix, the score matrix having a size of B x D or D x B, an i-th row or an i-th column in the score matrix storing D scores, and a j-th score being a matrix product of absolute values all weights in the i-th weight block and all mask values in the j-th mask block.

[0139] Steps (a) to (c) may be performed repeatedly until scores all mask blocks in the first sparse mask set give the i-th weight block is determined.

[0140] It can be seen that since any one element in a mask block takes the value of 0 or 1 , the score given to the i-th weight block by all mask blocks in the i-th sparse mask set corresponding to the i-th weight block is the sum of absolute values of weights selected from the i-th weight block by using elements taking the value of 1 in all mask blocks in the i-th sparse mask set, and the higher that score is, the better the sparsity training is.

[0141] In Step 3, an additional mask block giving the highest score to the i-th weight block is found, and a value of a counter corresponding to the i-th weight block and the additional mask block is incremented by 1.

[0142] In Step 4, weight blocks and mask blocks corresponding to E counters with E highest values in the counter matrix are determined as the E weight blocks and the E additional mask blocks, and designated indices of the E additional mask blocks are stored.

[0143] In S1102, the X weight blocks and the X additional mask blocks are determined in the one or more predefined durations.

[0144] In S1103, the X designated indices of the X additional mask blocks are assigned to the X weight blocks respectively.

[0145] That is to say, the X designated indices described above may be determined all in one predefined duration (E = X) or may be determined in batches over more predefined durations (E < X), e.g., the number (e.g., X / the total number of the predefined durations) or ratio of weight and mask blocks to be determined for each predefined duration may be set, which are not limited by embodiments of the present disclosure.

[0146] The above S1101 to S1104 may be performed repeatedly until B designated indices are determined for the B weight blocks respectively, in which an i-th designated index is an index of a mask block in the i-th sparse mask set that gives the i-th weight block a highest score for highest numbers of time.

[0147] It will be noted that if a designated index has been determined for the i-th weight block, the i-th weight block can be regarded as having been frozen, or the i-th weight block has been bound with a mask block corresponding to the designated index, or a mapping relationship or corresponding relationship has been established therebetween, i.e. , the mask block corresponding to the designated index can be used directly to generate the all-mask matrix in the sparse matrix multiplication operation, thereby reducing the workload of the matrix multiplication and improving the training efficiency.

[0148] In addition, if E = X = 0 in Schemes 1 to 3 described above, Schemes 2 and 3 are essentially the same as Scheme 1.

[0149] The following is described in conjunction with the example shown in FIG. 14.

[0150] In Step 1 above, it is assumed that all weight blocks are each of a size of 4 x 4 (K = P = 4, i.e., the weight blocks are all squares), that all weight blocks correspond to the first mask set, that there are a total of 90 types of mask blocks in the first sparse mask set that satisfy the 2:4 (N = 2 and M = 4) sparsity constraint on both by rows and by columns, and that, as shown in (a) and (b) of FIG. 14, the weight matrix is split into B weight blocks (B = 8), in which the 1-st weight block and the 6-th weight block correspond to mask blocks with designated indices of 48 and 75, respectively.

[0151] In Step 4 above, as shown in (c) of FIG. 14, except for the 1-st row and the 6-th row (corresponding to the 1-st weight block and the 6-th weight block, respectively) in the counter matrix, among the other 6 rows, a counter in the 1-st column and 4-th row and a counter in the 7-th row and 88-th column have the highest count values (6 and 5, respectively), then, as shown in (d) of FIG. 14, it is determined that a designated index of a mask block corresponding to the 4-th weight block and a designated index of a mask block corresponding to the 7-th weight block are 1 and 88, respectively.

[0152] It will be noted that in embodiments of the present disclosure, random number indices generated individually for different weight blocks, as well as multiple random number indices generated for the same weight block, are completely independent of each other, with the random ranges are each of [1,D].

[0153] Embodiments of the present disclosure provide a computer program product including an instruction that,when executed by a computing device, causes the computing device to realize the method described in the above method embodiments.

[0154] Embodiments of the present disclosure provide a computer-readable storage medium having a computer program or instruction stored thereon, when the computer program or instruction is run on a computing device, causing the computing device to perform the method described in the above method embodiments.

[0155] Embodiments of the present disclosure provide a chip system including a processing circuit and a storage medium. The storage medium has a computer program instruction stored thereon, when the computer program instruction is executed by the processing circuit, causing the chip system to implement the method provided by any of the embodiments.

[0156] It should be understood that a processor in embodiments of the present disclosure may be a central processing unit (CPU); alternatively, the processor may be another general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or another programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, or the like. The general purpose processor may be a microprocessor, or the processor may be any conventional processor.

[0157] It should also be understood that a memory in embodiments of the present disclosure may be volatile (transitory) memory or non-volatile (non-transitory) memory or may include both volatile and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (programmable ROM, PROM for short), an erasable programmable read-only memory (erasable PROM, EPROM for short), an electrically erasable programmable read-only memory (electrically EPROM, EEPROM for short), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. Many forms of random access memory (RAM) are available through exemplary but not restrictive explanations, such as static random access memory (static RAM, SRAM for short), dynamic random access memory (dynamic RAM, DRAM for short), synchronous dynamic random access memory (synchronous DRAM, SDRAM for short), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM for short), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM for short), synchlink dynamic random access memory (synchlink DRAM, SLDRAM for short) and a direct rambus random access memory (direct rambus RAM, DR RAM for short).

[0158] The embodiments described above may be implemented, in whole or in part, by software, hardware (e.g., circuitry), firmware, or any other combination thereof. When implemented using software, the above embodiments may be implemented, in whole or in part, in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When loading or executing the computer instructions or computer programs on a computer produces, in whole or in part, a process or function in accordance with the embodiments of the present disclosure. The computer may be a general purpose computer, a special-purpose computer, a computer network, or another programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, e.g., the computer instructions may be transmitted by wired or wireless (e.g., infrared, Wi-Fi, microwave, etc.) means from one website site, computer, server, or data center to another website site, computer, server, or data center. The computer-readable storage medium may be any usable medium to which a computer is capable of accessing or a data storage device such as a server, data center, etc. that contains a collection of one or more usable media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, and tape), an optical medium (e.g., DVD), or a semiconductor medium, where the semiconductor medium may be a solid state disk.

[0159] It should be understood that the term "and / or" herein is merely a description of the relationship of associated objects, and indicates that three types of relationships may exist, e.g., A and / or B, which may indicate the existence of A alone, the existence of both A and B, and the existence of B alone, where A and B may each be in singular or in plural. In addition, the character " / " herein, generally indicates that associated objects before and after this character are in an "or" relationship, but may also indicate in an "and / or" relationship, specific reference can bemade to the preceding and following text for understanding.

[0160] In embodiments of the present disclosure, "at least one" refers to one or more, and "a plurality of' refers to two or more. The expression "at least one item (one) of the following" or similar expressions refers to any combination of these items, including any combination of singular item (one) or plural items (plurality). For example, at least one item (one) of a, b, or c may be expressed as: a, b, c, a and b, a and c, b and c, or a, b and c, where a, b, and c may each be singular or plural.

[0161] In embodiments of the present disclosure, the words "exemplarily", "for example", and the like are used to denote examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" in the present disclosure should not be construed as being preferred or advantageous over other embodiments or design schemes. Rather, the use of the term "example" is intended to present concepts in a concrete manner.

[0162] It should be understood that in various embodiments of the present disclosure, the magnitude of the serial numbers of the above-described processes does not imply a sequential order of execution, and the order in which the processes are executed should be determined by their function and inherent logic, and should not constitute any limitation on the processes for implementing embodiments of the present disclosure.

[0163] Those of ordinary skill in the art may realize that the units and algorithmic steps of the various examples described in conjunction with the embodiments disclosed herein are capable of being implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the particular application and design constraints of the technical solution. The skilled professional may use different methods to implement the described functions for each particular application, but such implementations should not be considered outside the scope of the present disclosure.

[0164] Those skilled in the art can clearly understand that for the convenience and simplicity of description, the working processes of the systems, devices and modules described above may refer to the corresponding processes in the above embodiments for the method, and details will not be repeated here.

[0165] In embodiments provided by the present disclosure, it will be understood that the system, device and method disclosed may be implemented in other ways. For example, embodiments of the device described above are merely exemplary. For example, the division of the units is only a logical functional division. In actual implementation, there are another division manners. For example, a plurality of devices or components are combined or integrated into another system, or some features may be ignored or not executed. Moreover, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

[0166] The units described as separate components may or may not be physically separated, and a component shown as a unit may or may not be a physical unit, that is, the component may be located in one place, or may be distributed to multiple different network units. Part or all of the units may be selected according to actual needs to achieve the purpose of the solutions of the embodiments.

[0167] In addition, various functional units in various embodiments of the present disclosure may be integrated in a single processing unit, or each unit may physically exist separately, or two or more units may be integrated in a single unit.

[0168] The described functions may be stored in a computer-readable storage medium when implemented as a software functional unit and sold or used as a stand-alone product. Based on this understanding, the technical solution of the present disclosure is essentially or contributes to the prior art or parts of the technical solution may be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in various embodiments of the present disclosure. The aforementioned storage media includes a USB flash drive, a removable hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or optical disk, and other media that can store program codes.

[0169] The foregoing description is only specific embodiments of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or replacements that a person skilled in the artcould conceive of within the technical scope of the present disclosure shall Ise included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims

CLAIMS1. A neural network training method, the method comprising: splitting a weight matrix into B weight blocks, the B weight blocks having a same size and defining B sparse mask sets, each sparse mask set including D mask blocks satisfying N:M sparsity constraint, wherein N, M, B and D are all positive integers, N < M, B > 1, and D > 1; generating B random number indices respectively corresponding to the B weight blocks, the B random number indices each having a random range of [1 ,D]; determining B mask blocks using the B random number indices and the B sparse mask sets, the B mask blocks corresponding to the B weight blocks; generating a mask matrix using the B mask blocks; and implementing sparse matrix multiplication operation using the mask matrix.

2. The method according to claim 1 , wherein determining the B mask blocks using the B random number indices and the B sparse mask sets includes: randomly selecting B mask blocks for the B weight blocks from the B sparse mask sets respectively based on the B random number indices; obtaining X additional mask blocks, the X additional mask blocks being mask blocks that give X weight blocks in the B weight blocks a highest score for highest numbers of times in one or more predefined durations, wherein X is an integer and 0 X < B; and updating the B mask blocks by replacing X mask blocks of the B mask blocks selected from the B sparse mask sets with the X additional mask blocks; and wherein generating the mask matrix using the B mask blocks includes: generating the mask matrix using the updated B mask blocks, a position of an i-th mask block in the updated B mask blocks in the mask matrix coinciding with a position of an i-th weight block in the weight matrix, wherein i is a positive integer and 1 < i < B.

3. The method according to claim 2, wherein the i-th mask block in the mask matrix generated with the updated B mask blocks satisfies:Ai = Fi + (1 - sign(Fi)) * Ri; wherein Ai is an index of the i-th mask block in the mask matrix generated with the updated B mask blocks, Ri is an index of the i-th mask block of the B mask blocks selected from the B sparse mask sets, Fi is an index of a mask block corresponding to the i-th weight block in the X additional mask blocks, and sign (Fi) is an index sign function that satisfies:wherein Fi < 1 or Fi > D indicates that a mask block corresponding to the i-th weight block in the B weight blocks is not determined, and 1 < Fi < D indicates that the mask block corresponding to the i-th weight block in the B weight blocks is determined as a mask block with a designated index Fi in an i-th sparse mask set.

4. The method according to claim 1 , wherein determining the B mask blocks using the B random number indices and the B sparse mask sets includes: randomly selecting B-X mask blocks for B-X weight blocks from B-X sparse mask sets respectively based on B-X random number indices, wherein X is an integer and 0 < X < B; obtaining X additional mask blocks, the X additional mask blocks being mask blocks thatgive X weight blocks in the B weight blocks a highest score for highest numbers of times in one or more predefined durations; and updating the B mask blocks by merging the B-X mask blocks and the X additional mask blocks; and wherein generating the mask matrix using the B mask blocks includes: generating the mask matrix using the updated B mask blocks, a position of an i-th mask block in the updated B mask blocks in the mask matrix coinciding with a position of an i-th weight block in the weight matrix, wherein i is a positive integer and 1 < i < B.

5. The method according to claim 2 or 4, wherein obtaining the X additional mask blocks includes: obtaining the X additional mask blocks from X sparse mask sets in the B sparse mask sets based on X designated indices.

6. The method according to claim 5, the method further comprises: before obtaining the X additional mask blocks from the X sparse mask sets in the B sparse mask sets based on the X designated indices, determining X designated indices for X weight blocks in the B weight blocks respectively in the one or more predefined durations.

7. The method according to claim 6, wherein determining the X designated indices for the X weight blocks in the B weight blocks respectively in the one or more predefined durations includes: determining E weight blocks and E additional mask blocks in one predefined duration of the one or more predefined durations, the E additional mask blocks being mask blocks that give the E weight blocks a highest score for highest numbers of times in the one predefined duration, wherein E is an integer and 0 < E < X; determining the X weight blocks and the X additional mask blocks in the one or more predefined durations; and assigning the X designated indices of the X additional mask blocks to the X weight blocks respectively.

8. The method according to claim 7, wherein determining the E weight blocks and the E additional mask blocks includes: setting all elements in a counter matrix of a size of B * D and all elements in an index array of a size of B to 0, a counter in an i-th row and a j-th column in the count matrix being used for counting when a j-th mask block in the i-th sparse mask set gives the i-th weight block a highest score, an i-th element in the index array being used for storing a designated index of a mask block corresponding to the i-th weight block, wherein j is a positive integer and 1 < j < D; once every certain number of training steps, determining a score the j-th mask block in the i-th sparse mask set gives the i-th weight block; finding an additional mask block that gives the highest score to the i-th weight block, and incrementing a value of a counter corresponding to the i-th weight block and the additional mask block by 1; and determining weight blocks and mask blocks corresponding to E counters with E highest values in the counter matrix as the E weight blocks and the E additional mask blocks, and storing designated indices of the E additional mask blocks.

9. The method according to claim 8, wherein the B weight blocks are all sub-weight matrices of a size of K x P, wherein K and P are positive integers, and K = M and / or P = M; and the B sparse mask sets are all first sparse mask sets; wherein determining the score the j-th mask block in the i-th sparse mask set gives the i-th weight block includes: generating a reshaped matrix using the B weight blocks, the reshaped matrix having a sizeof B x (K * P) or (K * P) x B, and an i-th row or an i-th column of the reshaped matrix including absolute values of all weights of the i-th weight block in the B weight blocks; generating an all-mask matrix using a first sparse mask set, the all-mask matrix having a size of (K * P) x D or D x (K * P), a j-th column or a j-th row of the all-mask matrix including all mask values of a j-th mask block in the first sparse mask set, and positions of all mask values of the j-th mask block in the first sparse mask set in the all-mask matrix coinciding with positions of the absolute values of all weights of the i-th weight block in the reshaped matrix respectively; and obtaining a score matrix by performing matrix multiplication operation on the reshaped matrix and the all-mask matrix, the score matrix having a size of B x D or D x B, an i-th row or an i-th column in the score matrix storing D scores, and a j-th score being a matrix product of absolute values all weights in the i-th weight block and all mask values in the j-th mask block.

10. The method according to claim 1 , wherein determining the B mask blocks using the B random number indices and the B sparse mask sets includes: randomly selecting B mask blocks for the B weight blocks from the B sparse mask sets respectively based on the B random number indices; and generating the mask matrix using the B mask blocks includes: generating the mask matrix using the B mask blocks selected from the B sparse mask sets, a position of an i-th mask block of the B mask blocks in the mask matrix coinciding with a position of an i-th weight block in the weight matrix, wherein i is a positive integer and 1 < i < B.

11. A computing device, comprising: a memory; and at least one processor coupled to the memory; wherein the memory is configured to store computer instructions that, when executed by the at least one processor, cause the computing device to: split a weight matrix into B weight blocks, the B weight blocks having a same size and defining B sparse mask sets, each sparse mask set including D mask blocks satisfying N:M sparsity constraint, wherein N, M, B and D are all positive integers, N < M, B > 1 , and D > 1 ; generate B random number indices respectively corresponding to the B weight blocks, the B random number indices each having a random range of [1 ,D]; determine B mask blocks using the B random number indices and the B sparse mask sets, the B mask blocks corresponding to the B weight blocks; generate a mask matrix using the B mask blocks; and implement sparse matrix multiplication operation using the mask matrix.

12. The computing device according to claim 11, wherein the computer instructions, when executed by the at least one processor, cause the computing device to: randomly select B mask blocks for the B weight blocks from the B sparse mask sets respectively based on the B random number indices; obtain X additional mask blocks, the X additional mask blocks being mask blocks that give X weight blocks in the B weight blocks a highest score for highest numbers of times in one or more predefined durations, wherein X is an integer and 0 < X < B; update the B mask blocks by replacing X mask blocks of the B mask blocks selected from the B sparse mask sets with the X additional mask blocks; and generate the mask matrix using the updated B mask blocks, a position of an i-th mask block in the updated B mask blocks in the mask matrix coinciding with a position of an i-th weight block in the weight matrix, whereini is a positive integer and 1 < i < B.

13. The computing device according to claim 12, wherein the i-th mask block in the mask matrix generated with the updated B mask blocks satisfies:Ai = Fi + (1 - sign(Fi)) * Ri; wherein Ai is an index of the i-th mask block in the mask matrix generated with the updated B mask blocks, Ri is an index of the i-th mask block of the B mask blocks selected from the B sparse mask sets, Fi is an index of a mask block corresponding to the i-th weight block in the X additional mask blocks, and sign (Fi) is an index sign function that satisfies:(0 ifFi < 1 orFi > D sign(F>) = (1if i <Fi<D;wherein Fi < 1 orFi > D indicates that a mask block corresponding to the i-th weight block in the B weight blocks is not determined, and 1 < Fi < D indicates that the mask block corresponding to the i-th weight block in the B weight blocks is determined as a mask block with a designated index Fi in an i-th sparse mask set.

14. The computing device according to claim 11 , wherein the computer instructions, when executed by the at least one processor, cause the computing device to: randomly select B-X mask blocks for B-X weight blocks from B-X sparse mask sets respectively based on B-X random number indices, wherein X is an integer and 0 < X < B; obtain X additional mask blocks, the X additional mask blocks being mask blocks that give X weight blocks in the B weight blocks a highest score for highest numbers of times in one or more predefined durations; update the B mask blocks by merging the B-X mask blocks and the X additional mask blocks; and generate the mask matrix using the updated B mask blocks, a position of an i-th mask block in the updated B mask blocks in the mask matrix coinciding with a position of an i-th weight block in the weight matrix, wherein i is a positive integer and 1 < i < B.

15. The computing device according to claim 12 or 14, wherein the computer instructions, when executed by the at least one processor, cause the computing device to: obtain the X additional mask blocks from X sparse mask sets in the B sparse mask sets based on X designated indices.

16. The computing device according to claim 15, wherein the computer instructions, when executed by the at least one processor, cause the computing device to: before obtaining the X additional mask blocks from the X sparse mask sets in the B sparse mask sets based on the X designated indices, determine X designated indices for X weight blocks in the B weight blocks respectively in the one or more predefined durations.

17. The computing device according to claim 16, wherein the computer instructions, when executed by the at least one processor, cause the computing device to: determine E weight blocks and E additional mask blocks in one predefined duration of the one or more predefined durations, the E additional mask blocks being mask blocks that give the E weight blocks a highest score for highest numbers of times in the one predefined duration, wherein E is an integer and 0 < E < X; determine the X weight blocks and the X additional mask blocks in the one or more predefined durations; and assign the X designated indices of the X additional mask blocks to the X weight blocks respectively.

18. The computing device according to claim 16, wherein the computer instructions, when executed by the at least one processor, cause the computing device to:set all elements in a counter matrix of a size of B x D and all elements in an index array of a size of B to 0, a counter in an i-th row and a j-th column in the counter matrix being used for counting when a j-th mask block in the i-th sparse mask set gives the i-th weight block a highest score, an i-th element in the index array being used for storing a designated index of a mask block corresponding to the i-th weight block, wherein j is a positive integer and 1 £ j < D; once every certain number of training steps, determining a score the j-th mask block in the i-th sparse mask set gives the i-th weight block; find an additional mask block that gives the highest score to the i-th weight block, and incrementing a value of a counter corresponding to the i-th weight block and the additional mask block by 1; and determine weight blocks and mask blocks corresponding to E counters with E highest values in the counter matrix as the E weight blocks and the E additional mask blocks, and store designated indices of the E additional mask blocks.

19. The computing device according to claim 18, wherein the B weight blocks are all sub-weight matrices of a size of K « P, wherein K and P are positive integers, K = M and / or P = M; the B sparse mask sets are all first sparse mask sets; wherein the computer instructions, when executed by the at least one processor, cause the computing device to: generate a reshaped matrix using the B weight blocks, the reshaped matrix having a size of B x (K * P) or (K * P) x B, and an i-th row or an i-th column of the reshaped matrix including absolute values of all weights of the i-th weight block in the B weight blocks; generate an all-mask matrix using a first sparse mask set, the all-mask matrix having a size of (K * P) x D or D x (K * P), a j-th column or a j-th row of the all-mask matrix including all mask values of a j-th mask block in the first sparse mask set, and positions of all mask values of the j-th mask block in the first sparse mask set in the all-mask matrix coinciding with positions of the absolute values of all weights of the i-th weight block in the reshaped matrix respectively; and obtain a score matrix by performing matrix multiplication operation on the reshaped matrix and the all-mask matrix, the score matrix having a size of B D or D x B, an i-th row or an i-th column in the score matrix storing D scores, and a j-th score being a matrix product of the absolute values of all weights in the i-th weight block and all mask values in the j-th mask block.

20. The computing device according to claim 11 , wherein the computer instructions, when executed by the at least one processor, cause the computing device to: randomly select B mask blocks for the B weight blocks from the B sparse mask sets respectively based on the B random number indices; and generate the mask matrix using the B mask blocks selected from the B sparse mask sets, a position of an i-th mask block in the mask matrix coinciding with a position of an i-th weight block of the B mask blocks in the weight matrix, wherein i is a positive integer and 1 < i < B.

21. A computer-readable storage medium having stored thereon computer instructions that, when executed by a computer, cause the computer to implement the method described in any one of claims 1 to 10.

22. A computer program product containing instructions that, when executed by a computer, cause the computer to implement the method described in any one of claims 1 to 10.

23. A chip system, comprising a processing circuit and a storage medium, wherein the storage medium has stored thereon computer instructions that, when executed by the processing circuit, cause the chip system to implement the method described in any one of claims 1 to 10.

Images