A method and system for analyzing tensor structure information

Abstract

The application discloses a tensor structure information analysis method and system, and the method comprises the following steps: for an input neural network application program, based on program semantic analysis, the structure information of each tensor in the program is analyzed, a description tensor corresponding to the structure information of the tensor is created, the description tensor is converted into a nested list, and a tree structure is constructed based on the nested list; the tree structure is traversed, all node paths are recorded, a constant tensor index description list is formed, based on the constant tensor index description list, the index description list of a non-constant tensor is obtained through program semantic analysis; and a structure index information optimization mechanism is used to merge and optimize the obtained constant tensor and non-constant tensor index description list. Through the index description of the tensor structure information and the capture of the structure information and operation characteristics from the semantics, the application improves the structure characteristic description precision, and greatly improves the utilization rate of the computing resources and the computing efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and program analysis, in particular to a neural network tensor structure information analysis method and system. BACKGROUND

[0002] With the significant effects of neural network models in various tasks, researchers are constantly committed to developing more complex and diverse models. In order to fully utilize the computational potential of these complex models, existing neural network frameworks (such as PyTorch) generally use many highly optimized operators. However, as the complexity and flexibility of the model increases, the waste of computing resources also becomes increasingly significant. For example, many masking operations will cause unnecessary computational redundancy, reducing the efficiency of model inference and training. Therefore, how to effectively perceive and optimize the computational overhead brought by these semantic operations has become a key problem in neural network model optimization.

[0003] Existing neural network frameworks fail to deeply analyze and optimize various neural network applications written by users, especially programs containing complex semantics. The use of masks increases the diversity of neural network semantics and the robustness of neural network performance, enabling neural networks to more flexibly adapt to the needs of different tasks. On the other hand, it objectively causes a large amount of computational redundancy, especially in practical applications. When masks are widely used, the system may process a lot of invalid data during the computation process, increasing unnecessary computational burden. If developers do not use the specific coding paradigm provided by the framework, especially when writing custom operations or optimization strategies, it is often difficult to avoid performing some unnecessary operations, which leads to a significant decrease in computational efficiency.

[0004] In addition, these redundant calculations not only waste computing resources, but also can cause significant delays in the model inference and training process, especially in resource-constrained embedded devices or edge computing environments, which seriously affects the actual deployment and application of neural networks.

[0005] Existing neural network frameworks often fail to deeply analyze and optimize complex neural network programs written by users, especially programs containing complex semantics. Although these frameworks enhance the flexibility and adaptability of neural networks by supporting diverse syntax and operations, such as masking to increase the diversity of semantics and improve the robustness of neural networks, these flexible operations also pose significant challenges to computational efficiency.

[0006] Masking operations can make neural networks better handle different task requirements, but often lead to a large amount of computational redundancy in practical applications. The existing technical problems of redundancy mainly include:

[0007] 1. Invalid data calculation: When masks are widely used, the system needs to handle a large amount of invalid data during the calculation process. These data do not produce actual value during the calculation process, but consume valuable computing resources and increase the calculation burden;

[0008] 2. Decreased calculation efficiency: If the developer does not follow the specific coding paradigm provided by the framework, especially when writing custom operations or optimization strategies, it often leads to the execution of many unnecessary operations. This directly leads to a significant decrease in calculation efficiency, affecting the inference and training process of the model.

[0009] These calculation redundancies not only waste computing resources, but also can cause significant inference and training delays. This problem is particularly prominent in resource-constrained environments, such as embedded devices or edge computing devices, severely affecting the actual deployment and application effect of neural network models.

[0010] On this basis, several frameworks have proposed methods to optimize neural network models with sparse structures, however, the initial setting of tensor information structure, especially sparse information structure, is highly dependent on manual experience. SparTA and other frameworks need to manually set the sparse structure information of input tensors and weights before running. These optimization premise settings bring additional burden to neural network programming.

[0011] Therefore, in order to solve the technical problem that the above optimization brings additional burden to neural network programming, solve the challenge that it is difficult to effectively capture and utilize tensor structure information in custom neural network programs in the prior art, and the technical problem that the capture difficulty of tensor structure information and the description difficulty of tensor structure information cannot be solved, it is urgent to propose a method and system capable of perceiving tensor structure information through static analysis, capable of realizing tensor structure information analysis based on program semantic analysis; it is urgent to propose a system capable of in-depth analysis and optimization of neural network programs while maintaining the diversity and flexibility of neural network semantic expression, reducing calculation redundancy and improving overall efficiency. SUMMARY

[0012] In order to solve the challenge that it is difficult to effectively capture and utilize tensor structure information in custom neural network programs in the prior art, the present application proposes a tensor structure information analysis system based on program semantic analysis.

[0013] In a first aspect, the embodiments of the present application provide a tensor structure information analysis method, the method comprising:

[0014] Index description list calculation step: for the input neural network application program, based on program semantic analysis, the structure information of each tensor in the program is analyzed, the description tensor of the structure information of the tensor is created, the description tensor is converted into a nested list, and a tree structure is constructed based on the nested list;

[0015] The tensor structure information semantic analysis step includes: traversing the tree structure, recording all node paths, forming a constant tensor index description list, and obtaining an index description list of a non-constant tensor based on the constant tensor index description list through program semantic analysis.

[0016] The index list optimization step includes: using a structure index information optimization mechanism to merge and optimize the constant tensor and the non-constant tensor index description list.

[0017] In the embodiment of the present application, before the index description list calculation step, the program representation parsing step further includes:

[0018] The program representation parsing step includes: parsing the source code of the program to obtain the specified initial value of all constant tensors, and creating a structure information description tensor corresponding to the constant tensor, wherein the description tensor has the same shape as the original tensor, and the value of each position of the description tensor is used to mark the specific value or precision information of the corresponding original tensor element.

[0019] The initialization tensor structure information description step includes: determining which specific value corresponding description tensor is obtained according to the application program setting, setting the value of the corresponding position of the description tensor to 0 for the position with a value of 0 in the original tensor, and setting the value of the corresponding position of the description tensor to the preset specific value for the position with a value of the preset specific value.

[0020] In the embodiment of the present application, the index description list calculation step includes:

[0021] The tree structure construction step includes: converting the description tensor into a nested list to construct a tree structure, wherein each layer of the tree corresponds to a dimension of the tensor, and the node of any layer represents the index information of the corresponding any dimension.

[0022] The list compaction step includes: starting the compaction operation from the innermost layer of the tree structure, and the compaction operation is: if all elements in the nested list are the preset specific value, the nested list is replaced by the preset specific value.

[0023] The iteration processing step includes: iterating from the inner layer of the tree structure to the outer layer dimension, and performing the compaction operation on each layer, and if the elements of a list are not all preset specific values, the compaction operation is not performed.

[0024] The index recording step includes: traversing the tree to record all node paths that satisfy that the sub-tree is the preset specific value, and forming an index description list of the preset specific value.

[0025] In the embodiment of the present application, the tensor structure information semantic analysis step further includes:

[0026] Assignment operation analysis step: when there is an assignment statement in the program, the constant i-th row j-th column is set to 0, then an item [i,j] is added to the index description list with a value of 0, wherein i and j represent the number of rows and columns of the matrix respectively;

[0027] Specific structure tensor identification step: for a two-dimensional tensor created by the program with a diagonal line of a specific value scale, the length and width of the two-dimensional tensor are both l, then the array (0, 0), (1, 1), …, (l, l) is added to the index description list corresponding to scale.

[0028] In specific embodiments of the present application, the above tensor structure information semantic analysis step further comprises:

[0029] Region uniform value identification and setting step: region uniform value identification and setting, by identifying the region with a uniform value in the tensor, setting the uniform value region to a uniform specific value;

[0030] Specific mode operation feature mining step: for index positions with a specific interval mode, establish the corresponding calculation template, define the step size and offset, and calculate the positions separated by the step size and offset;

[0031] Effective operation ratio calculation step: the effective operation ratio is the ratio of the number of non-zero values in each innermost dimension to the dimension length, and different operation modes are corresponded according to the effective operation ratio, and the operation mode is extracted by semantic analysis or set by the program.

[0032] In specific embodiments of the present application, the above index list optimization step further comprises:

[0033] Node merging step: starting from the deepest leaf node, traversing the tree upwards, if all child node indexes of a certain node are in the record and cover all positions of the corresponding dimension, merging the child nodes and shrinking the subtree to a node, so that the node becomes a leaf node;

[0034] Iteration simplification step: repeatedly execute the node merging step, merge layer by layer upwards, until it cannot be simplified any more, and obtain the most simplified index description list.

[0035] In a second aspect, embodiments of the present application provide a tensor structure information analysis system, which adopts the tensor structure information analysis method as described above, and the system comprises:

[0036] Index description list calculation module: for the input neural network application program, based on program semantic analysis, analyze the structure information of each tensor in the program, create a structure information description tensor corresponding to the tensor, convert the description tensor into a nested list, and build a tree structure based on the nested list;

[0037] The tensor structure information semantic analysis module is configured to traverse the tree structure, record all node paths, form a constant tensor index description list, and obtain an index description list of a non-constant tensor based on the constant tensor index description list and through program semantic analysis.

[0038] The index list optimization module is configured to adopt a structure index information optimization mechanism to merge and optimize the constant tensor and the non-constant tensor index description list.

[0039] In the embodiment of the present application, the index description list calculation module is further provided with:

[0040] The program representation analysis module is configured to parse the source code of the program to obtain the specified initial value of all constant tensors, and create a structure information description tensor corresponding to the constant tensor, wherein the description tensor has the same shape as the original tensor, and the value of each position of the description tensor is used to mark the specific value or precision information of the corresponding original tensor element.

[0041] The initialization tensor structure information description module is configured to determine which specific value corresponding description tensor is obtained according to the application program setting, set the value of the corresponding position of the description tensor to 0 for the position with 0 value in the original tensor, and set the value of the corresponding position of the description tensor to the preset specific value for the position with the preset specific value.

[0042] In a third aspect, the embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to realize the steps of the tensor structure information analysis method.

[0043] In a fourth aspect, the embodiment of the present application provides an electronic device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the program to realize the steps of the tensor structure information analysis method.

[0044] Compared with the related prior art, the present application has the following outstanding beneficial effects:

[0045] 1) The method of the present application designs a structure index information optimization mechanism, which reduces redundant records by merging and simplifying the index list, and improves the precision and storage efficiency of the index description.

[0046] 2) The structure index information optimization mechanism proposed in the method of the present application not only reduces the redundant storage of structure information, but also significantly improves the information reading and processing speed during calculation. In a complex neural network program, through the optimization processing of the structure characteristics, the system can more efficiently perceive and process the tensor structure information, and ensure that the data processing and storage performance in the program execution process reaches the optimum. BRIEF DESCRIPTION OF DRAWINGS

[0047] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0048] Figure 1 The schematic diagram of the tensor structure information analysis method of the embodiment of the application;

[0049] Figure 2 The schematic diagram of the tensor structure information analysis method of the embodiment of the application;

[0050] Figure 3 The schematic diagram of the tensor structure information analysis system of the embodiment of the application;

[0051] Figure 4 The schematic diagram of the computer hardware of the application. DETAILED DESCRIPTION

[0052] It should be noted that the processor of the application is the control center of the electronic device, which can be one processor or a plurality of processing elements. For example, it can be one or more central processing units (CPU), application specific integrated circuits (ASIC), or one or more integrated circuits configured to implement the embodiments of the application, such as one or more digital signal processors (DSP) or one or more field programmable gate arrays (FPGA).

[0053] Optionally, the processor can execute various functions of the electronic device by running or executing software programs stored in the memory and calling data stored in the memory.

[0054] In a specific implementation, as an embodiment, the processor can include one or more CPUs. Each of these processors can be a single core processor or a multi-core processor. The processor here can refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions). The electronic device can include a server, a desktop computer, a notebook computer, a smart phone, a tablet computer, an embedded computer, etc., wherein the embedded computer includes vehicles and robots, etc.

[0055] The memory is used to store a software program for implementing the scheme of the application, and is controlled by the processor to perform, and the specific implementation can refer to the method embodiments described above, which will not be repeated here.

[0056] It should be noted that the structure of the electronic device shown in the drawings of the present application does not constitute a limitation thereon, and the actual knowledge structure recognition device can include more or fewer components than shown, or combine certain components, or different component arrangements.

[0057] The above embodiments can be implemented, wholly or partially, by software, hardware (such as a circuit), firmware, or any other combination. When implemented by software, the above embodiments can be implemented, wholly or partially, in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the flow or function described in the embodiments of the present application is wholly or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another, for example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wired (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, data center, etc. containing one or more available medium collections. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state disk.

[0058] It should also be understood that the term "and / or" herein merely describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. In addition, the character " / " herein generally represents that the front and rear associated objects are in an "or" relationship, but can also represent an "and / or" relationship, which can be understood in the context.

[0059] In the present application, "at least one" means one or more, and "multiple" means two or more. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can represent a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0060] It should also be understood that the size of the sequence of the above processes does not mean the order of execution, the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the application.

[0061] In several embodiments provided by the present application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the above-described device embodiments are merely schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0062] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0063] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically independently, or two or more units can be integrated into one unit.

[0064] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the present application that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of software products. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk and various program code storage media.

[0065] In order to make the above features and effects of the present application more clear and easy to understand, the following embodiments are specifically described below, and the detailed description is made in conjunction with the accompanying drawings. The present disclosure discloses one or more embodiments comprising the features of the present application. The disclosed embodiments are only for illustration. The scope of protection of the present application is not limited to the disclosed embodiments, and the present application is defined by the appended claims.

[0066] The following is a system embodiment corresponding to the above method embodiment. The present embodiment can be implemented in cooperation with the above embodiments. The related technical details mentioned in the above embodiments are still valid in the present embodiment. In order to reduce repetition, they will not be described again here. Correspondingly, the related technical details mentioned in the present embodiment can also be applied in the above embodiments.

[0067] In view of the tensor information capturing problem of user-defined neural network programs, the present application aims to propose a tensor structure information analysis system based on program semantic analysis. The system aims to solve the challenge of effectively capturing and utilizing tensor structure information in user-defined neural network programs in the prior art. The tensor structure information in neural network programs is usually implicitly in the complex program semantics, and this information is crucial for optimizing computational efficiency. In order to better optimize the execution efficiency of user-defined neural network programs, the system needs to deeply analyze the program semantics, identify various structured information of tensors, and combine the specific semantic properties of operators for corresponding optimization processing.

[0068] The system provides as much useful information as possible for the optimization process by perceiving the structural information on the result tensor that can be analyzed under the general semantic expression and the specific structure caused by the operator with special semantics. By embedding these structural information into the computation graph, the system not only optimizes the execution efficiency of individual operators, but also realizes the overall optimization of the computation graph in a global range. The ability to comprehensively perceive and label structural information enables the system to greatly reduce unnecessary computational load and improve the execution efficiency of neural network programs in various hardware environments, especially in resource-constrained scenarios.

[0069] In this process, the main technical difficulties include:

[0070] 1. Difficulty in capturing tensor structure information: User-defined neural network programs usually have highly complex and dynamic structures, and the sparsity of tensors and other structural information are often implicitly in complex program semantics. It is very difficult to extract these implicit information directly from program code, and it is necessary to deeply analyze the semantics and execution logic of the program, accurately identify the operations of tensors and their impact on the structure. This involves in-depth analysis of the abstract syntax tree of neural network programs and various intermediate expression forms to capture the structural characteristics of tensors.

[0071] 2. Difficulty in describing tensor structure information: Accurately and systematically describing the tensor structure information captured from program semantics is also a major challenge. A general and efficient representation method is needed to convert the captured sparse information into a form that can represent the sparse structure of tensors. This description method must be able to cover all possible structural features so that subsequent optimization processes can effectively utilize this information, while also avoiding redundancy and improving the efficiency of information processing.

[0072] The following detailed description is provided in conjunction with specific embodiments:

[0073] Example 1

[0074] like Figure 1 As shown, Figure 1 This is a schematic flowchart of the tensor structure information analysis method of the present invention. An embodiment of this application provides a method for analyzing tensor structure information, the method comprising:

[0075] Index description list calculation step 101: For the input neural network application, based on program semantic analysis, analyze the structural information of each tensor in the program, create a description tensor of the structural information of the tensor, convert the description tensor into a nested list, and build a tree structure based on the nested list;

[0076] Tensor structure information semantic analysis step 102: Traverse the tree structure, record all node paths, form a constant tensor index description list, and obtain the non-constant tensor index description list through program semantic analysis based on the constant tensor index description list.

[0077] Index list optimization step 103: Use the structure index information optimization mechanism to merge and optimize the obtained constant tensor and non-constant tensor index description lists.

[0078] 1) Capturing neural network tensor structure information based on program semantics

[0079] Current neural network systems often exhibit numerous structural features, such as sparsity, when processing tensors on a large scale. Under certain operational modes, zero elements or low-impact weights contribute little to the actual computation, or are even completely unnecessary. However, traditional computational methods fail to fully utilize this information, leading to a waste of computational resources. Existing neural network compilers and optimization systems primarily rely on shallow structural analysis of operators or computational graphs, or heavily depend on manual annotation and optimization, failing to automatically delve into the semantic level of the program for comprehensive identification of sparse structures. Therefore, intelligently acquiring and annotating the structural information of tensors in neural network programs has become one of the key optimization problems.

[0080] To solve this problem, the present application proposes a tensor structure information capturing mechanism based on program semantic analysis. By analyzing the semantic information of the neural network program, the system can deeply understand the tensor operation and its results, thereby accurately capturing the structural features. By analyzing the structured information of each tensor in the program, the system can identify semantics such as all-zero elements, mask operations, and generate corresponding structured information. These structured information not only provides optimization basis for subsequent calculation, but also provides complete basic data for the system to perceive inefficient calculation path.

[0081] Specifically, the present application generates a companion matrix and index description list for describing the tensor structure features by automatically analyzing the program code, and details the corresponding specific information in the tensor. The system can identify operators with mask semantics in the user-defined neural network program, zero element regions of tensors, and structural features of temporary tensors generated by code logic, and also supports user-defined operation features. By recording these information, it can further ensure that the program can accurately know the structure of the tensor during execution.

[0082] 2) Index-based neural network tensor structure feature description

[0083] After obtaining the tensor structure information, how to efficiently represent and store these information becomes another important technical problem for optimizing system performance. Existing methods usually only record individual structural features of tensors, resulting in high complexity of structural information storage, lack of generalization, and inability to achieve efficient description and calculation. In complex neural networks, as the dimension and tensor size increase, the processing efficiency of structural information will decrease significantly, and how to simplify and optimize these information becomes a challenge.

[0084] To solve this problem, the present application designs a structure index information optimization mechanism, which combines and simplifies the index list, reduces redundant records, and improves the accuracy and storage efficiency of index description. Specifically, when the system captures the sparse structure of the tensor, it will merge the same or similar elements in the structure. For example, if all elements in a dimension are of the same precision, the system will merge these elements into a unified description, thereby reducing storage space and reducing the index difficulty of sparse regions.

[0085] The structure index information optimization mechanism in the present application not only reduces the redundant storage of structural information, but also significantly improves the information reading and processing speed during calculation. In complex neural network programs, by optimizing the processing of structural features, the system can more efficiently perceive and process tensor structure information, ensuring that the data processing and storage performance during program execution is optimal.

[0086] In summary, the application solves the technical problems of tensor structure information acquisition and description, realizes comprehensive perception and efficient processing of tensor structure information in the neural network program, and provides strong support for subsequent optimization.

[0087] Embodiment two

[0088] As Figure 2 shown, the embodiment of the application provides a tensor structure information analysis method, the method comprising:

[0089] Program representation analysis step 201: analyzing the source code of the program to obtain the specified initial value of all constant tensors, and creating a structure information description tensor corresponding to the constant tensor, wherein the description tensor has the same shape as the corresponding original tensor, and the value of each position of the description tensor is used to mark the specific value or precision information of the corresponding original tensor element;

[0090] Initialization tensor structure information description step 202: determining which specific value corresponding description tensor is obtained according to the application program setting, setting the value of the description tensor corresponding position to 0 for the position with value 0 in the original tensor, and setting the value of the description tensor corresponding position to the preset specific value for the position with the preset specific value.

[0091] Index description list calculation step 203: based on program semantic analysis, the structure information of each tensor in the program is analyzed, the description tensor of the structure information of the tensor is created, the description tensor is converted into a nested list, and a tree structure is constructed based on the nested list.

[0092] Tensor structure information semantic analysis step 204: traversing the tree structure, recording all node paths, forming a constant tensor index description list, and based on the constant tensor index description list, obtaining the index description list of the non-constant tensor through program semantic analysis.

[0093] Index list optimization step 205: using a structure index information optimization mechanism to merge and optimize the obtained constant tensor and non-constant tensor index description list.

[0094] In the embodiment of the application, the above index description list calculation step 203 comprises:

[0095] Tree structure construction step: converting the description tensor into a nested list to construct a tree structure, wherein each layer of the tree corresponds to a dimension of the tensor, and the node of any layer represents the index information of the corresponding any dimension;

[0096] List compaction step: starting from the innermost dimension of the tree structure to perform a compaction operation, and the compaction operation is: if all elements in the nested list are preset specific values, the nested list is replaced by the preset specific values.

[0097] Iterative processing steps: Iterate from the inner layer of the tree structure to the outer layer in turn, and perform a compaction operation on each layer. If the elements of a list are not all of the preset specific values, the compaction operation is not performed.

[0098] Indexing steps: By traversing the tree, record all node paths that satisfy the condition that all subtrees have preset specific values, forming an index description list with preset specific values.

[0099] In a specific embodiment of this invention, the index description of tensor structure information is presented through deep semantic analysis of tensor operations in neural network programs, proposing a systematic method to accurately capture the structure of tensors. The system identifies 0 and 1 elements in tensors by default, and the index description for capturing a fixed value of `num` can be set using the `setCatch Const(num, flag)` API. Furthermore, the system captures the structured sparsity introduced by masking operations and the structure information of temporary tensors generated by user-defined program logic. By generating a companion matrix and an index description list, the system annotates the tensor's structure information in detail and uses this information for subsequent computational processing.

[0100] Let's assume that the shapes of tensors S and C are (d1, d2, ..., dc). n ), where d i (i = 1, 2, ..., n) represents the length of the i-th dimension. First, create a nested list describing the specific value 'a' corresponding to the original tensor. Then, create a tensor of shape (d1, d2, ..., d...). n A list of length d, whose innermost element is a list of length d. n The list, d n Let be the number of tensors with a specific value 'a'. If the original tensor has the desired value 'a', then set it to 'a'; otherwise, set it to 0. The second-to-last layer contains 'd'. n-1 The length of the innermost layer is d. nThe list... the outermost list has d1 sublists. Suppose a list is l; if all elements inside l are specific values ​​'a', then l is replaced with 'a', or l is compressed to 'a'. Thus, the parent list may become a new flat list represented entirely by 'a'. This compression operation is performed for each deepest list iteration. If not all elements inside a list are 'a', no compression operation is performed. This nested list is essentially equivalent to a tree structure, which we can call T. Its node value i represents the index i of the information structure represented by its subtree in the current dimension, and the subtree corresponding to a node at height h represents the structural information in the h-th dimension. Assuming the original tensor S has dimension n, then a node at height h represents all structural information in the h-th dimension, and the maximum depth of its subtree is nh. By breadth-first traversing the tree, the paths corresponding to all nodes satisfying the following conditions are recorded to form an index: i→j means that the path corresponding to the node with value i at depth 1 and value j at depth 2 is the index of the current value 'a', and all leaf nodes of its corresponding subtree are 'a'. By obtaining a list of all such indices, the index description for a specific value 'a' can be obtained. By default, the index description for a specific value 0 of each tensor is calculated. The generation of index descriptions for 0 can be canceled using `setCatch Const(0, false)`, and the generation of index descriptions for tensor 'a' can be enabled by setting `setCatch Const(a, true)`.

[0101] Furthermore, after generating the sparse index list through semantics, for an index list, starting from the list item with the deepest nesting depth, the nested list is considered as a tree structure. The h-th level corresponds to a node in the h-th dimension of the tensor. If the recorded index value is i, then its subtree represents the structural information of the i-th position of the h-th dimension tensor. For leaf nodes, assuming the values ​​on the path to the current node (including the leaf node itself) are (i1, i2, ..., i... h ), then represents S[i1][i2]…[i] of the source tensor S. n The subtensors corresponding to ] are all 0. Consider the following process: starting from the parent node of the deepest leaf node (level j), traverse the subtree, and whenever all child nodes of the subtree can fill 0…d j When a subtree is shrunk to a single node, that is, when all the child nodes of a parent node can guarantee that all values ​​of all child tensors are 0, all the child nodes of that node are deleted, making that node a leaf node. This process is iterated to obtain the simplest index description.

[0102] Technical effects:

[0103] 1. Improve the accuracy of structural feature description: By generating index description corresponding to the tensor structure for each required value such as 0, 1, etc., the system can accurately label the structural features contained in the tensor for each specific value required, ensuring that important structural information is not missed during the calculation process.

[0104] 2. Improve the efficiency of calculation: By generating the index description list, the system can quickly locate the specific structural information in the tensor, reducing redundant calculation operations. Using the structured sparse feature and the compacted index description, the system can efficiently skip unnecessary elements in the subsequent calculation process, significantly shortening the calculation time and optimizing the overall running performance of the neural network.

[0105] In the embodiment of the present application, the above tensor structure information semantic analysis step 204 further comprises:

[0106] Assignment operation analysis step: when there is an assignment statement in the program, the constant i-th row j-th column is set to 0, then add an item [i,j] to the index description list with value 0, where i and j represent the number of rows and columns of the matrix respectively;

[0107] Specific structure tensor recognition step: for the two-dimensional tensor created by the program with a diagonal line of a specific value scale, the length and width of the two-dimensional tensor are both l, then add the array (0,0), (1,1), …, (l,l) to the index description list corresponding to scale.

[0108] In the embodiment of the present application, the above tensor structure information semantic analysis step 204 further comprises:

[0109] Region uniform value recognition and setting step: region uniform value recognition and setting, by recognizing the region with uniform value in the tensor, setting the uniform value region to a specific value;

[0110] Specific pattern operation feature mining step: for the index position with a specific interval pattern, establish the corresponding calculation template, define the step size and offset, and calculate the positions with the step size and offset;

[0111] Effective operation ratio calculation step: the effective operation ratio is the ratio of the non-zero value in each innermost dimension to the length of the dimension, according to the effective operation ratio corresponding to different operation modes, the operation mode is extracted by semantic analysis or set by the program.

[0112] In the specific embodiments of the present application, the structural information and operation characteristics are captured from semantics. On the basis of the sparse index list obtained for the constant tensor known at the beginning of all programs, the sparse index list of all non-constant tensors can be analyzed through program semantics. For the semantics of setting the (i, j) dimension of a certain tensor A to 0 (which usually occurs in the pruning process, i.e., directly pruning a certain channel, which can be the jth channel of the ith batch), i.e., the semantics of A[i][j] = 0, an item [i, j] is added to the index list corresponding to the 0 element. For example, if the program creates a two-dimensional tensor with a length and width of l and a diagonal line with a specific value scale, (0, 0), (1, 1), …, (l, l) are added to the index description corresponding to scale (if there is no description, a description is created). The information is scalable and highly compressed, and lays a solid foundation for runtime analysis and optimization.

[0113] By identifying the region with uniform values in the tensor (which is usually important for matrix blocking), special processing can be performed on these regions during the calculation. For a region with all a certain value a, the value can be directly used in the calculation to avoid calculating each element one by one. A specific setRegionValue((i, j, …), (x1, y1), (x2, y2), a) API is used to set the region with the upper left corner (x1, y1) and the lower right corner (x2, y2) in the sub-tensor with index (i, j, …) to a. In particular, for sparse operation mode, setRegionSparse((i, j, …), (x1, y1), (x2, y2), true) can be used to set the region with the upper left corner (x1, y1) and the lower right corner (x2, y2) in the sub-tensor with index (i, j, …) to a, and the true flag indicates that the sub-tensor is 0 except for this region, and false indicates that the values of the sub-tensor except for this region are not set.

[0114] For index positions with specific interval patterns, corresponding calculation templates can be established, and only necessary positions are calculated. This can be achieved by defining the stride and offset. Starting from the first address represented by the current index (i, j, …), every t positions are calculated, and m consecutive data are invoked setPatternStride((i, j, …), offset, t, m, a). When m is 1, it degenerates into the classic unstructured sparse. At the same time of setting, the proportion p of non-zero values in the length of each innermost dimension can be counted, also known as the effective operation proportion. If the effective operation number of all one-dimensional dimensions of a two-dimensional matrix is 0.5 after setting or detection (using getPatternStride((i, j, …))), it means that only half of the original tensor size is operated on each dimension, corresponding to the 2:4 operation mode. Special operation modes can be extracted through semantic analysis or set by the program.

[0115] Technical effects:

[0116] 1. Improve the utilization rate of computing resources: the optimization calculation of specific regions and modes reduces unnecessary calculation operations and improves the utilization efficiency of computing resources.

[0117] 2. Improve the calculation efficiency: the optimized index description can speed up the reading and processing of structure information, ensure that the structure information in the calculation process can be efficiently utilized, and avoid unnecessary waste of computing resources.

[0118] In the specific embodiment of the present application, the above index list optimization step 205 further comprises:

[0119] Node merging step: starting from the deepest leaf node, traverse the tree upwards, if all child nodes of a node are in the record and cover all positions of the corresponding dimension, merge the child nodes and shrink the subtree into a node, so that the node becomes a leaf node;

[0120] Iterative simplification step: repeatedly execute the node merging step, merge layer by layer upwards until it cannot be simplified, and get the simplest index description list.

[0121] The following uses Python source code as an embodiment of the present application to illustrate the method of the present application:

[0122] Step 1: Program representation parsing

[0123] Take Python source code as input, parse its abstract syntax tree (AST) or intermediate representation (IR), and generate a data flow graph corresponding to the source code. By traversing the AST or IR, the definitions, assignments and operations of all tensors in the program are obtained, laying the foundation for subsequent structure information capture.

[0124] Step 2: Initialize the structural information description of tensors

[0125] For all constant tensors with specified initial values parsed in Step 1, create corresponding structural information description tensors D. The description tensors have the same shape as the original tensors S, and each position's value is used to mark the specific value or precision information of the corresponding source tensor element. According to the API setting decision, determine which specific values correspond to the description tensors, and by default, obtain the descriptions corresponding to 0 and 1. For positions with value 0 in the source tensor, set the value of the corresponding position in the description tensor to 0; for positions with a specific non-zero constant a, set the value of the corresponding position in the description tensor to a.

[0126] Step 3: Index description list calculation

[0127] Convert the description tensors into nested lists and construct a tree structure T. Each layer of the tree corresponds to a dimension of the tensor, and the nodes of the h-th layer represent the index information of the h-th dimension. The specific process is as follows

[0128] List compaction: Starting from the innermost dimension, if all elements in a certain list are a specific value a, replace the list with a.

[0129] Iterative processing: Iteratively process the outer dimensions one by one, and try to perform the above compaction operation on each layer. If the elements of a certain list are not all a, do not perform compaction.

[0130] Index recording: Record all node paths that satisfy the condition that the sub-tree is full of specific values a by breadth-first traversal of the tree T, forming an index description list. For example, the path [i, j] represents the position with index i in the first dimension and index j in the second dimension, and its corresponding sub-tensor is full of specific values a.

[0131] Step 4: Semantic analysis of tensor structure information

[0132] Based on the capture of constant tensor structure information, further capture the structure information of non-constant tensors through semantic analysis of the program, which includes:

[0133] Assignment operation analysis: When there is an assignment statement in the program, such as A[i][j] = 0, add an item [i, j] to the index description list with value 0. This usually occurs during pruning, where a certain channel is directly pruned.

[0134] Specific structure tensor identification: For a two-dimensional tensor created by the program with a diagonal line of specific values scale, add (0, 0), (1, 1), …, (l, l) to the corresponding index description, where l is the size of the tensor. This information is extensible and highly compressed, providing a solid foundation for runtime analysis and optimization.

[0135] Step 5: Identification and setting of region uniform value

[0136] By identifying the region with uniform value in the tensor, special processing can be performed on these regions. SetRegionValue is used to set the region uniform specific value, and setRegionSparse is used to set the sparse specific region.

[0137] Step 6: Operation feature mining of specific pattern

[0138] For index positions with specific interval patterns, corresponding calculation templates are established, and only necessary positions are calculated. By defining the stride and offset, setPatternStride API is used to set in the program.

[0139] Step 7: Merge and simplify of index description

[0140] The index description list obtained in steps 3, 4, 5 and 6 is merged and optimized:

[0141] Node merging: Starting from the deepest leaf node, traverse the tree T upwards. If all child nodes of a node are in the record and cover all positions in that dimension, merge these child nodes to make the node a leaf node.

[0142] Iterative simplification: Repeat the above process, merge layer by layer upwards until no further simplification is possible, and obtain the simplest index description list.

[0143] Example three

[0144] As shown in Figure 3 , the application embodiment provides a tensor structure information analysis system, which adopts the above-mentioned tensor structure information analysis method, and the system comprises:

[0145] A program representation parsing module 301 is configured to parse the source code of the program to obtain specified initial values of all constant tensors, and create a structure information description tensor corresponding to the constant tensors, wherein the description tensor has the same shape as the original tensor, and the value of each position of the description tensor is used to mark the specific value or precision information of the corresponding original tensor element.

[0146] An initialization tensor structure information description module 302 is configured to determine which specific value corresponding description tensor is obtained according to the application program setting, set the value of the corresponding position of the description tensor to 0 for the position with value 0 in the original tensor, and set the value of the corresponding position of the description tensor to the preset specific value for the position with the preset specific value.

[0147] The index description list calculation module 303 is configured to analyze the structural information of each tensor in the program based on program semantic analysis, create a structural information description tensor corresponding to the tensor, convert the description tensor into a nested list, and build a tree structure based on the nested list for the input neural network application program.

[0148] The tensor structural information semantic analysis module 304 is configured to traverse the tree structure, record all node paths, form a constant tensor index description list, and obtain a variable tensor index description list based on the constant tensor index description list through program semantic analysis.

[0149] The index list optimization module 305 is configured to merge and optimize the obtained constant tensor and variable tensor index description lists by using a structural index information optimization mechanism.

[0150] Embodiment four

[0151] The embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the steps of the tensor structural information analysis method.

[0152] Embodiment five

[0153] The embodiment of the present application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the tensor structural information analysis method when executing the program.

[0154] In addition, the tensor structural information analysis method of the embodiment of the present application can be implemented by an electronic device, such as a computer device. Figure 1 The tensor structural information analysis method of the embodiment of the present application described above can be implemented by an electronic device, such as a computer device. Figure 4 The hardware structure of the computer device according to the embodiment of the present application is shown in the figure.

[0155] In some embodiments, the computer device can further include a communication interface 83 and a bus 80. As shown in the figure, the processor 81, the memory 82, and the communication interface 83 are connected through the bus 80 and complete communication with each other. Figure 4

[0156] Specifically, the processor 81 can include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or can be configured as one or more integrated circuits implementing the embodiment of the present application.

[0157] The memory 82 can be used to store or cache various data files required for processing and / or communication, and possible computer program instructions executed by the processor 81.​

[0158] The processor 81 realizes the analysis method of the tensor structure information in any of the above embodiments by reading and executing the computer program instructions stored in the memory 82.

[0159] The technical features of the above embodiments can be combined in any manner. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described, but it should be understood that any combination of the technical features is within the scope of the present disclosure as long as the combination does not result in contradictions.

[0160] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the patent scope of the present application. It should be pointed out that, for those skilled in the art, some modifications and improvements can be made without departing from the concept of the present application, and these are within the protection scope of the present application. Therefore, the patent protection scope of the present application should be subject to the appended claims.

Claims

1. A method of analyzing tensor structure information, characterized by, The method comprises: a program representation analysis step: analyzing the source code of the program to obtain the specified initial value of all constant tensors, and creating a structure information description tensor corresponding to the constant tensor, wherein the description tensor has the same shape as the corresponding original tensor, and the value of each position of the description tensor is used to mark the specific value or precision information of the corresponding original tensor element; an index description list calculation step: based on program semantic analysis, analyzing the structure information of each tensor in the program, creating a description tensor of the structure information of the tensor, converting the description tensor into a nested list, and constructing a tree structure based on the nested list for an input neural network application program; a tensor structure information semantic analysis step: traversing the tree structure, recording all node paths, forming a constant tensor index description list, and obtaining an index description list of a non-constant tensor based on the constant tensor index description list through program semantic analysis; The tensor structure information semantic analysis step further comprises: Assignment operation analysis step: when there is an assignment statement in the program, the constant i-th row j-th column is set to 0, then add an item to the index description list with value 0 where i and j represent the number of rows and columns of the matrix, respectively; an index list optimization step: merging and optimizing the obtained constant tensor and non-constant tensor index description lists by using node merging and iteration optimization.

2. The method of claim 1, wherein the tensor structure information is analyzed by using a tensor decomposition method. Before the index description list calculation step, there is further included: an initialization tensor structure information description step: determining which description tensor corresponding to a specific value is obtained according to the application program setting, setting the value of the corresponding position of the description tensor to 0 for the position with a value of 0 in the original tensor, and setting the value of the corresponding position of the description tensor to the preset specific value for the position with a value of the preset specific value.

3. The method of claim 1 or 2, wherein the tensor structure information is analyzed by using a method of analyzing a tensor structure information. The index description list calculation step comprises: a tree structure construction step: converting the description tensor into a nested list to construct a tree structure, wherein each layer of the tree corresponds to one dimension of the tensor, and the node of any layer represents the index information of the corresponding any dimension; a list compaction step: starting from the innermost dimension of the tree structure to perform a compaction operation, wherein the compaction operation is to replace the nested list with the preset specific value if all elements in the nested list are the preset specific value; an iteration processing step: iterating from the inner layer to the outer layer dimension of the tree structure, and performing the compaction operation for each layer, if the elements of a list are not all the preset specific value, the compaction operation is not performed; an index recording step: recording all node paths that satisfy that the sub-tree is all the preset specific value by traversing the tree to form an index description list of the preset specific value.

4. The method of claim 1 or 2, wherein the tensor structure information is analyzed by using a method of analyzing a tensor structure information. The tensor structure information semantic analysis step further comprises: Specific structure tensor identification step: for the two-dimensional tensor created by the program with the diagonal line of specific value scale, the length and width of the two-dimensional tensor are Then add the array to the corresponding index description list.

5. The method of claim 1 or 2, wherein the tensor structure information is analyzed by using a method of analyzing a tensor structure information. The tensor structure information semantic analysis step further comprises: a region uniform value identification and setting step: identifying and setting a region uniform value by identifying a region with a uniform value in the tensor, and setting a region uniform specific value for the region with the uniform value; a specific mode operation feature mining step: establishing a corresponding calculation template for an index position with a specific interval mode, defining a step size and an offset, and performing calculation on positions spaced by the step size and the offset; The step of calculating the effective operation proportion includes: calculating the proportion of non-zero values in each innermost dimension to the dimension length; and determining different operation modes according to the effective operation proportion, wherein the operation modes are extracted by semantic analysis or set by a program.

6. The method of claim 1 or 2, wherein the tensor structure information is analyzed by using a method of analyzing a tensor structure information. The step of optimizing the index list further includes: The step of node merging includes: starting from the deepest leaf node, traversing the tree upwards, merging the nodes if all child nodes of a node are in the record and cover all positions of the corresponding dimension, shrinking the sub-tree into a node, and making the node a leaf node; The step of iteration simplification includes: repeatedly performing the step of node merging, merging layer by layer upwards, until no further simplification is possible, and obtaining the simplest index description list.

7. A system for analyzing tensor structure information, employing the method for analyzing tensor structure information according to any one of claims 1 to 6, characterized by The system includes: A program representation analysis module: analyzing the source code of the program to obtain the specified initial value of all constant tensors, and creating a structure information description tensor corresponding to the constant tensor, wherein the description tensor has the same shape as the original tensor, and the value of each position of the description tensor is used to mark the specific value or precision information of the corresponding original tensor element; An index description list calculation module: used for analyzing the structure information of each tensor in the program based on program semantic analysis for an input neural network application program, creating a structure information description tensor corresponding to the tensor, converting the description tensor into a nested list, and constructing a tree structure based on the nested list; A tensor structure information semantic analysis module: used for traversing the tree structure, recording all node paths, forming a constant tensor index description list, and obtaining an index description list of a non-constant tensor based on the constant tensor index description list through program semantic analysis; The tensor structure information semantic analysis module further includes: Assignment operation analysis module: when there is an assignment statement in the program, the constant i-th row j-th column is set to 0, then add an item in the index description list with value 0 where i and j represent the number of rows and columns of the matrix, respectively; An index list optimization module: used for merging and optimizing the obtained constant tensor and non-constant tensor index description lists by using a structure index information optimization mechanism.

8. The system for analyzing tensor structure information according to claim 7, wherein, Before the index description list calculation module, there is further included: The program representation analysis module is configured to parse source code of the program to obtain specified initial values of all constant tensors, and create structure information description tensors corresponding to the constant tensors wherein the description tensors have the same shape as the corresponding original tensors, and a value at each position of the description tensors is used to mark a specific value or precision information of an element of the corresponding original tensor. An initialization tensor structure information description module: used for determining which specific value corresponding description tensor is obtained according to the application program setting, setting the value of the description tensor corresponding to the position of the original tensor with a value of 0 as 0, and setting the value of the description tensor corresponding to the position with a value of a preset specific value as the preset specific value.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by a processor to implement the steps of the tensor structure information analysis method in any one of claims 1-6.

10. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the steps of the tensor structure information analysis method in any one of claims 1-6.

Images