Data structure conversion method and electronic device

Abstract

The embodiments of the present disclosure provide a data structure conversion method and an electronic device. The method includes: obtaining a first data structure; performing a first structure conversion on the first data structure based on a preset rule to determine a second data structure; obtaining thread information of a hardware execution unit, wherein the thread information is related to the total number of multiple threads; and performing a second structure conversion on the second data structure based on the thread information to determine a third data structure, wherein the third data structure is used for the hardware execution unit to access at least one element in the first data structure through multiple threads. The computing performance of neural network operations can be improved.

Description

Technical Field

[0001] The present disclosure relates to a data structure conversion method and an electronic device. Background Art

[0002] In neural network computing technology, a high degree of coordination is required between the computing model, storage device, and multiple hardware execution circuits that actually perform parallel computing. The higher the coordination between the computing model, storage device, and hardware execution circuit, the more significantly the computing performance of the neural network computing can be improved. However, the logical structure of the data used by the computing model may be different from the configuration of the hardware execution circuit, resulting in low computing performance of the neural network computing. Summary of the invention

[0003] The present disclosure provides a data structure conversion method and an electronic device, which can improve the computing performance of neural network computing.

[0004] An embodiment of the present disclosure provides a data structure conversion method, which is used in an electronic device. The electronic device includes a hardware execution unit, the hardware execution unit has multiple threads, and the data structure conversion method includes: obtaining a first data structure, wherein the first data structure includes multiple elements; based on a preset rule, performing a first structure conversion on the first data structure to determine a second data structure, wherein a first arrangement of the multiple elements in the first data structure is different from a second arrangement of the multiple elements in the second data structure; obtaining thread information of the hardware execution unit, wherein the thread information is related to the total number of the multiple threads; and based on the thread information, performing a second structure conversion on the second data structure to determine a third data structure, wherein the third data structure is used for the hardware execution unit to access at least one of the multiple elements through the multiple threads, and a third arrangement of the multiple elements in the third data structure is different from the second arrangement.

[0005] The embodiment of the present disclosure further provides an electronic device, which includes a processor, a plurality of hardware operation circuits and a storage circuit. The processor is connected to the plurality of hardware operation circuits and the storage circuit. The processor is used to execute the data structure conversion method.

[0006] Based on the above, after obtaining the first data structure, a first structure conversion can be performed on the first data structure based on a preset rule to determine a second data structure. On the other hand, thread information of the hardware execution unit can be obtained. In particular, the thread information is related to the total number of multiple threads. Based on the thread information, a second structure conversion can be performed on the second data structure to determine a third data structure. In particular, the third data structure can be used for the hardware execution unit to access at least one element in the first data structure through the multiple threads. Thereby, the problem of low computing performance of neural network operations, which is traditionally caused by the logical structure of the data used in the computing model being different from the configuration of the hardware execution circuit, can be improved, thereby improving the computing performance of the neural network operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 is a schematic diagram of an electronic device according to an embodiment of the present disclosure;

[0008] Figure 2 is a schematic diagram of data structure conversion according to an embodiment of the present disclosure;

[0009] Figure 3 is a schematic diagram of a third data structure according to an embodiment of the present disclosure;

[0010] Figure 4 is a schematic diagram of access control information according to an embodiment of the present disclosure;

[0011] Figure 5 is a schematic diagram showing how access control information is generated according to an operation instruction and a third data structure according to an embodiment of the present disclosure; and

[0012] Figure 6 is a flowchart of a data structure conversion method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0013] As mentioned above, with the gradual popularization of artificial intelligence technology, the performance requirements for the computing model used as the brain of artificial intelligence and the hardware equipment that actually performs the calculations continue to increase in order to provide users with a better operating experience. However, the problem that may be encountered in practice is that even if the computing model used as the brain of artificial intelligence is continuously optimized and the performance of the hardware equipment that actually performs the calculations is continuously improved, if the problem of low computing performance of neural network calculations caused by the logical structure of the data used in the computing model being different from the configuration of the hardware execution circuit cannot be solved, the computing performance of neural network calculations will not be significantly improved.

[0014] The data structure conversion method and electronic device proposed in the embodiment of the present disclosure can convert the logical structure of the data to be provided to the hardware execution circuit for operation through a specially designed conversion mechanism. As a result, the problem of low computing performance of neural network operation caused by the logical structure of the data used in the operation model being different from the configuration of the hardware execution circuit can be improved, thereby improving the computing performance of the neural network operation.

[0015] Reference will now be made in detail to exemplary embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numerals are used in the drawings and the description to refer to the same or like parts.

[0016] Figure 1 is a schematic diagram of an electronic device according to an embodiment of the present disclosure. Figure 1 , please refer to Figure 1 The electronic device 10 may be a smart phone, a tablet computer, a notebook computer, a desktop computer, an industrial computer, a server, or a vehicle-mounted computer, and other electronic devices with computing and information processing functions, and the type of the electronic device 10 is not limited thereto.

[0017] The electronic device 10 includes a processor 11, a storage circuit 12, and a computing device 13. The processor 11 is connected to the storage circuit 12 and the computing device 13. The processor 11 can be used to be responsible for the overall or partial operation of the electronic device 10. For example, the processor 11 may include a central processing unit (CPU), or other programmable general-purpose or special-purpose microprocessors, digital signal processors (DSP), programmable controllers, application specific integrated circuits (ASIC), programmable logic devices (PLD), or other similar devices or combinations of these devices.

[0018] The storage circuit 12 may be used to store data. For example, the storage circuit 12 may include a volatile storage circuit and a non-volatile storage circuit. The volatile storage circuit is used to store data volatilely. For example, the volatile storage circuit may include a random access memory (RAM) or a similar volatile storage medium. The non-volatile storage circuit is used to store data non-volatilely. For example, the non-volatile storage circuit may include a read-only memory (ROM), a solid state disk (SSD), a traditional hard disk drive (HDD) or a similar non-volatile storage medium.

[0019] The computing device 13 can be used to provide physical computing and information processing functions. For example, the computing device 13 may include a graphics processing unit (GPU), a general-purpose computing on a graphics processing unit (GPGPU), a video processing unit (VPU), a neural-network processing unit (NPU), a tensor processing unit (TPU), or other hardware devices with computing and information processing functions, and the type of the computing device 13 is not limited thereto. In addition, the number of the computing devices 13 may be one or more, and the present disclosure is not limited thereto.

[0020] In one embodiment, the computing device 13 is disposed inside the electronic device 10. Figure 1 In one embodiment, the computing device 13 may also be disposed outside the electronic device 10 and may be connected to the electronic device 10 via a wired or wireless method, which is not limited in the present disclosure.

[0021] In one embodiment, the storage circuit 12 can be used to store the computing model 101. The computing model 101 may include at least one computing model of a machine learning model, a deep learning model, a neural network model, various decision models, various mathematical models, and various algorithm models, and the type of the computing model 101 is not limited thereto. The computing model 101 can be implemented by at least one of a program code, an instruction code, and a machine code or other types of various functional description languages. In addition, the number of the computing model 101 can be one or more, and the present disclosure is not limited thereto.

[0022] In one embodiment, the computing device 13 can be used in conjunction with the computing model 101. In one embodiment, the processor 11 can run the computing model 101 to provide tasks related to one or more operations (also referred to as computing tasks). The processor 11 can provide the computing tasks to the computing device 13 through at least one operation instruction. The internal circuit of the computing device 13 can execute the computing tasks provided by the processor 11 according to the operation instruction, and return the computing results to the processor 11.

[0023] In one embodiment, the computing device 13 includes hardware execution units 13(1)-13(n). The hardware execution units 13(1)-13(n) can be used to execute the computing tasks indicated by the operation instructions separately or in parallel and return the computing results to the processor 11. It should be noted that the total number of the hardware execution units 13(1)-13(n) can be adjusted according to practical needs and is not limited by the present disclosure.

[0024] In one embodiment, the processor 11 may obtain a data structure (also referred to as a first data structure). For example, the processor 11 may obtain the first data structure according to the computational requirements of the computational model 101. The first data structure may include a plurality of elements.

[0025] In one embodiment, the processor 11 may perform a structural transformation (also referred to as a first structural transformation) on the first data structure based on a preset rule to determine (e.g., generate) a transient data structure (also referred to as a second data structure). The second data structure also includes the plurality of elements originally included in the first data structure.

[0026] It should be noted that the arrangement of the plurality of elements in the first data structure (also referred to as the first arrangement) is different from the arrangement of the plurality of elements in the second data structure (also referred to as the second arrangement). For example, the arrangement of the plurality of elements in the first data structure (i.e., the first arrangement) may be a two-dimensional array, and the arrangement of the plurality of elements in the second data structure (i.e., the second arrangement) may be a one-dimensional array.

[0027] In one embodiment, the processor 11 may obtain thread information of the hardware execution unit 13(i). For example, the hardware execution unit 13(i) may be any one of the hardware execution units 13(1) to 13(n). The thread information is related to the total number of multiple threads in the hardware execution unit 13(i). For example, the thread information may reflect the total number of multiple threads in the hardware execution unit 13(i). For example, assuming that the hardware execution unit 13(i) includes "8" threads, the thread information may reflect that the total number of threads in the hardware execution unit 13(i) is "8". However, the present disclosure does not limit the total number of threads contained in any one of the hardware execution units 13(1) to 13(n).

[0028] In one embodiment, after obtaining the thread information of the hardware execution unit 13(i), the processor 11 may perform another structural transformation (also referred to as a second structural transformation) on the second data structure based on the thread information to determine (e.g., generate) another data structure (also referred to as a third data structure). In particular, the third data structure may be used for the hardware execution unit 13(i) to access at least one element in the first data structure through multiple threads within the hardware execution unit 13(i).

[0029] It should be noted that the arrangement of the plurality of elements in the third data structure (also referred to as the third arrangement) is also different from the arrangement of the plurality of elements in the second data structure (i.e., the second arrangement). For example, similar to the first data structure, the arrangement of the plurality of elements in the third data structure (i.e., the third arrangement) may be a two-dimensional array. However, the aspect ratio of the first data structure may be different from the aspect ratio of the third data structure.

[0030] In one embodiment, assuming that the arrangement of multiple elements in the first data structure (i.e., the first arrangement) is a p×q matrix, the arrangement of the multiple elements in the second data structure (i.e., the second arrangement) may be a 1×r matrix, and the arrangement of the multiple elements in the third data structure (i.e., the third arrangement) may be an s×t matrix. Parameters p, q, r, s, and t are all positive integers, and the parameters p, q, r, s, and t need to meet the requirements related to the above description. In one embodiment, p is different from s, q is different from t, the value of p×q is equal to r, and the value of s×t is also equal to r.

[0031] Figure 2 is a schematic diagram of data structure conversion according to an embodiment of the present disclosure. Figure 1 and Figure 2 In one embodiment, the first data structure 21 is composed of elements A to P. For example, the total number of elements A to P is 16. The arrangement of the elements A to P in the first data structure 21 (ie, the first arrangement) is a 4×4 matrix.

[0032] In one embodiment, the processor 11 may perform a structural transformation (i.e., a first structural transformation) on the first data structure 21 based on a preset rule to determine (e.g., generate) the second data structure 22. It should be noted that the second data structure 22 also includes the elements A to P in the first data structure 21. However, the arrangement of the elements A to P in the second data structure 22 (i.e., a second arrangement) is a 1×16 matrix (i.e., a one-dimensional matrix).

[0033] In one embodiment, the processor 11 may perform the first structure conversion based on a Shape:Strides function. According to the Shape:Strides function, the processor 11 may map (or arrange) the elements A-P originally arranged in the first data structure 21 (i.e., a two-dimensional array) to specific positions in the second data structure 22 in a column-first manner.

[0034] In one embodiment, the skip function can be expressed as ((4, 2), 2): (1, 4), 8). According to the skip function, the processor 11 can determine the four element combinations "A, E, I, M", "B, F, J, N", "C, G, K, O" and "D, H, L, P" along the vertical axis direction, taking four elements in the first data structure 21 as a group (with a step length of 1); then, along the horizontal axis direction, taking two elements in the first data structure 21 as a group (with a step length of 4), the four element combinations are recombined into two element combinations "A, E, I, M, B, F, J, N" and "C, G, K, O, D, H, L, P"; finally, with a step length of 8, the two element combinations are recombined into "A, E, I, M, B, F, J, N, C, G, K, O, D, H, L, P" in the second data structure 22. It should be noted that the skip function may also be adjusted according to practical requirements, and the present disclosure is not limited thereto.

[0035] In one embodiment, after determining (e.g., generating) the second data structure 22, the processor 11 may perform a structural transformation (i.e., a second structural transformation) on the second data structure 22 based on the thread information of the hardware execution unit 13(i) to determine (e.g., generate) the third data structure 23. For example, the processor 11 may sequentially arrange the elements A to P in the second data structure 22 to specific positions in the third data structure 23 based on the thread information of the hardware execution unit 13(i) (e.g., the total number of threads in the hardware execution unit 13(i)).

[0036] It should be noted that in Figure 2 In the embodiment of FIG. 1 , it is assumed that the hardware execution unit 13 (i) includes 8 threads. However, in other embodiments, the total number of threads included in the hardware execution unit 13 (i) may also be adjusted according to practical requirements, and the present disclosure is not limited thereto.

[0037] Figure 3 is a schematic diagram of a third data structure according to an embodiment of the present disclosure. Figure 2 and Figure 3, assuming that the hardware execution unit 13(i) includes threads (0) to (7) (i.e., the total number of threads in the hardware execution unit 13(i) is "8"), the first direction (i.e., the horizontal axis direction) of the third data structure 23 corresponds to threads (0) to (7). In addition, assuming that the hardware execution unit 13(i) includes multiple registers, the second direction (i.e., the vertical axis direction) of the third data structure 23 corresponds to at least one of the multiple registers. Thereafter, the hardware execution unit 13(i) can access an element (i.e., at least one of the elements A to P) at any position in the third data structure 23 through its internal thread (j) and register (k).

[0038] In one embodiment, the processor 11 may calculate the positions of the elements A-P in the third data structure 23 based on the following formulas (1.1) and (1.2).

[0039] j = z%w (1.1)

[0040] k=z / w (1.2)

[0041] In formulas (1.1) and (1.2), parameter z represents the zth position of elements A~P in the second data structure 22, parameter w represents the total number of threads included in the hardware execution unit 13(i), and operator % represents the remainder. For example, according to formula (1.1), the processor 11 can calculate that the elements "A, E, I, M, B, F, J, N" in the second data structure 22 correspond to threads (0)~thread (7) (i.e., j=0~7), and the elements "C, G, K, O, D, H, L, P" in the second data structure 22 also correspond to threads (0)~thread (7) (i.e., j=0~7). In addition, according to formula (1.2), the processor 11 can calculate that the elements "A, E, I, M, B, F, J, N" in the second data structure 22 correspond to register (0) (i.e., k=0), and the elements "C, G, K, O, D, H, L, P" in the second data structure 22 correspond to register (1) (i.e., k=1). Therefore, according to formulas (1.1) and (1.2), the processor 11 can determine that any one of the elements A~P is located at the intersection between the corresponding thread (j) and register (k). For example, in the third data structure 23, element A is located at the intersection between thread (0) and register (0), and element C is located at the intersection between thread (0) and register (1), and so on.

[0042] In one embodiment, when determining (eg, generating) a third data structure (eg, Figure 2 or Figure 3After obtaining the third data structure 23 of the computing model 101, the processor 11 can obtain the operation instruction. For example, the operation instruction can be provided by the computing model 101 (or the processor 11 itself). The operation instruction can instruct to perform a specific operation (also called the first operation) on at least one element (also called the first element) in the first data structure. For example, the first operation can include one or more logical operations, depending on the computing requirements of the computing model 101. The processor 11 can generate access control information corresponding to the first element based on the operation instruction and the third data structure. Then, the hardware execution unit 13 (i) can access the first element according to the access control information.

[0043] In one embodiment, the access control information includes thread index information and register index information. The thread index information is used to indicate the thread corresponding to the first element. For example, the thread index information may include a plurality of control bits, and the plurality of control bits may correspond to the plurality of threads respectively. The register index information is used to indicate the register corresponding to the first element.

[0044] Figure 4 is a schematic diagram of access control information according to an embodiment of the present disclosure. Figure 4 The access control information 41 includes register index information 411 and thread index information 412. The register index information 411 includes a control bit b1. For example, the control bit b1 may correspond to Figure 3 The thread index information 412 includes control bits b2(1) to b2(8). For example, the control bits b2(1) to b2(8) may correspond to Figure 3 In one embodiment, the combination of control bits b2(1)-b2(8) may be considered as a mask corresponding to threads (0)-thread (7). The hardware execution unit 13(i) may determine, locate and / or access at least one element (i.e., the first element) in the third data structure 23 according to the access control information 41.

[0045] Figure 5 is a schematic diagram showing the generation of access control information according to an operation instruction and a third data structure according to an embodiment of the present disclosure. Figure 5 , the processor 11 may obtain the operation instruction 51. For example, the operation instruction 51 may instruct to perform a first operation on the element “F, J, G, K” (ie, the first element) in the first data structure 21.

[0046] It should be noted that the operation instruction 51 may describe the element "F, J, G, K" based on the arrangement of the elements A-P in the first data structure 21 (i.e., the first arrangement). For example, the operation instruction 51 may carry access information. The access information may indicate that the first operation is performed on the element "F, J, G, K" (i.e., the first element) located at the coordinate position (1.1) and within the coordinate range of 2×2 in the first data structure 21.

[0047] After obtaining the operation instruction 51, the processor 11 may input the access information (e.g., the coordinate position (1.1) and the coordinate range of 2×2) carried by the operation instruction 51 into the conversion function 52 corresponding to the third data structure 23. For example, the conversion function 52 may be designed with reference to the conversion rules of the first structure conversion and the second structure conversion. Then, the processor 11 may determine the access control information 53 according to the output of the conversion function 52. For example, the access control information 53 may include the first access control information 531 and the second access control information 532. The first access control information 531 includes "b1=0" and "b2(1)~b2(8)="0, 0,0, 0, 0, 1, 1, 0". The second access control information 532 includes "b1=1" and "b2(1)~b2(8)="0, 1, 1, 0, 0, 0, 0, 0".

[0048] After obtaining the access control information 53, the hardware execution unit 13 (i) can access the elements "F" and "J" in the third data structure 23 according to the first access control information 531, and access the elements "G" and "K" in the third data structure 23 according to the second access control information 532. Then, the hardware execution unit 13 (i) can perform the first operation on the elements "F, J, G, K". In other words, through the conversion of the conversion function 52, the processor 11 can determine the position of at least one element (e.g., element "F, J, G, K") originally located in the first data structure 21 indicated by the operation instruction 51 in the third data structure 23. Then, through the access control information 53, the hardware execution unit 13 (i) can quickly determine, locate and / or access the elements "F, J, G, K", and then quickly complete the first operation performed on the elements "F, J, G, K".

[0049] Figure 6 is a flow chart of a data structure conversion method according to an embodiment of the present disclosure. Figure 6In step S601, a first data structure is obtained, wherein the first data structure includes a plurality of elements. In step S602, based on a preset rule, a first structural transformation is performed on the first data structure to determine a second data structure, wherein a first arrangement of the plurality of elements in the first data structure is different from a second arrangement of the plurality of elements in the second data structure. In step S603, thread information of a hardware execution unit is obtained, wherein the thread information is related to the total number of threads in the hardware execution unit. In step S604, based on the thread information, a second structural transformation is performed on the second data structure to determine a third data structure, wherein the third data structure is used for the hardware execution unit to access at least one of the plurality of elements through the plurality of threads, and a third arrangement of the plurality of elements in the third data structure is different from the second arrangement.

[0050] However, Figure 6 The steps in the above are described in detail, so I will not repeat them here. It is worth noting that Figure 6 Each step in the above process can be regarded as a plurality of program codes or circuits, which is not limited by the present disclosure. Figure 6 The method can be used in conjunction with the above exemplary embodiments or can be used alone, and the present disclosure is not limited thereto.

[0051] In summary, the data structure conversion method and electronic device proposed in the embodiment of the present disclosure can convert the logical structure of the data to be provided to the hardware execution circuit for operation through a specially designed conversion mechanism. As a result, the problem of low computing performance of neural network operation caused by the logical structure of the data used in the operation model being different from the configuration of the hardware execution circuit can be improved, thereby improving the computing performance of the neural network operation.

[0052] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A data structure conversion method, characterized in that: For an electronic device, wherein the electronic device includes a hardware execution unit, the hardware execution unit has multiple threads, and the data structure conversion method includes: Obtaining a first data structure, wherein the first data structure includes a plurality of elements; Based on a preset rule, performing a first structural transformation on the first data structure to determine a second data structure, wherein a first arrangement of the plurality of elements in the first data structure is different from a second arrangement of the plurality of elements in the second data structure, and the second arrangement is a one-dimensional array; Obtaining thread information of the hardware execution unit, wherein the thread information is related to the total number of the plurality of threads; and Based on the thread information, performing a second structure conversion on the second data structure to determine a third data structure, wherein the third data structure is used for the hardware execution unit to access at least one of the plurality of elements through the plurality of threads, and a third arrangement of the plurality of elements in the third data structure is different from the second arrangement, The first direction of the third data structure corresponds to the multiple threads, and the second direction of the third data structure corresponds to multiple registers. 2 . The data structure conversion method according to claim 1 , wherein the first arrangement method and the third arrangement method are both two-dimensional arrays.

3. The data structure conversion method according to claim 1, further comprising: Obtaining an operation instruction, wherein the operation instruction instructs to perform a first operation on at least one first element in the first data structure; generating access control information corresponding to the at least one first element according to the operation instruction and the third data structure; as well as The hardware execution unit accesses the at least one first element according to the access control information.

4. The data structure conversion method according to claim 3, wherein the access control information includes thread index information and register index information, the thread index information is used to indicate the thread corresponding to the at least one first element, and the register index information is used to indicate the register corresponding to the at least one first element. 5 . The data structure conversion method according to claim 4 , wherein the thread index information comprises a plurality of control bits, and the plurality of control bits correspond to the plurality of threads respectively.

6. The data structure conversion method according to claim 3, wherein the step of generating the access control information corresponding to the at least one first element according to the operation instruction and the third data structure comprises: inputting the access information carried by the operation instruction into a conversion function corresponding to the third data structure; as well as The access control information is determined according to the output of the conversion function. 7 . The data structure conversion method according to claim 3 , wherein the operation instruction describes the at least one first element based on the first arrangement.

8. An electronic device, characterized in that: The invention comprises a processor, a hardware operation circuit and a storage circuit, wherein the processor is connected to the hardware operation circuit and the storage circuit, and the processor is used to execute the data structure conversion method described in any one of claims 1-7.

Images