An operator computation method, apparatus, device and system

Abstract

The application provides an operator calculation method, device, equipment and system in the field of artificial intelligence. The method comprises the following steps: acquiring parameter data of a first data shape of an AI network, the first data shape being a data length in each dimension supported by the AI network for processing, the parameter data comprising combination information of at least two calculation units, the data supported by each calculation unit for processing being data with a second data shape, and the data length of each calculation unit in any dimension after combination according to the combination information being greater than or equal to the data length of the first data shape in the same dimension; and calling the at least two calculation units to perform calculation on first target data with the first data shape. Therefore, the application supports the change of the data shape in any range by combining the at least two calculation units, realizes the AI dynamic shape operator function, and improves the AI network starting speed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and particularly relates to an operator calculation method, device, equipment and system. BACKGROUND

[0002] AI (Artificial Intelligence) is to use a digital computer or a machine controlled by a digital computer to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a branch of computer science, which tries to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making. The research in the field of artificial intelligence includes robots, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, AI basic theory, etc.

[0003] At present, with the continuous development of computer technology, AI networks have been widely applied. Moreover, AI networks are becoming more and more complex, and the types of AI operators in AI networks are also increasing. Even for the same type of AI operator, the data shapes of the data that need to be supported for processing are also increasing. If an AI operator needs to be recompiled for each different data shape, the compilation will be more and more time-consuming, and the AI network startup speed will be reduced. SUMMARY

[0004] Embodiments of the present application provide an operator calculation method, device, equipment and system, which support the change of data shape in any range by combining at least two calculation units, so as to realize the AI dynamic shape operator function and improve the AI network startup speed.

[0005] In a first aspect, embodiments of the present application provide an operator calculation method, which comprises:

[0006] Obtaining parameter data of a first data shape of an AI network, the first data shape being the data length in each dimension supported by the AI network for processing, and the parameter data comprising combination information of at least two calculation units, the data supported by each calculation unit for processing being data with a second data shape, and the data length of each calculation unit in any dimension after combination according to the combination information being greater than or equal to the data length of the first data shape in the same dimension;

[0007] Calling the at least two calculation units to perform calculation on first target data with the first data shape.

[0008] In other words, for the first data shape that the AI ​​network can process, instead of recompiling the operator, the parameter data of the first data shape is obtained. This parameter data includes the combination information of at least two computing units. The calculation of the first target data with the first data shape is realized by calling these computing units. This avoids the need to recompile an AI operator for each different first data shape. Instead, it supports the change of data shape in any range by combining at least two computing units, realizes the AI ​​dynamic shape operator function, and improves the startup speed of the AI ​​network.

[0009] In this method, the number of first data shapes may be one or more. Since the first data shape actually refers to an operator's attribute—that is, the data length in each dimension that each operator in one or more operators in the AI ​​network supports processing—the number of first data shapes may be one or more. These multiple operators can be operators of the same type or operators of different types. For operators of the same type, if the operators support different data shapes, the number of first data shapes may be multiple; for operators of different types, the different operator types will also result in a possible number of first data shapes.

[0010] The second data shape refers to the data length in each dimension that the computing unit can process.

[0011] The relationship between the second data shape and the first data shape is as follows: the data length of the second data shape of each computing unit in any dimension after being combined according to the combined information is greater than or equal to the data length of the first data shape in the same dimension.

[0012] For example, the second data shape includes three dimensions: length, width, and height. After the second data shape of each computational unit is combined according to the aforementioned combination information, the data length in the length dimension is greater than or equal to the data length of the first data shape in the length dimension; the data length in the width dimension is greater than or equal to the data length of the first data shape in the width dimension; and the data length in the height dimension is greater than or equal to the data length of the first data shape in the height dimension. The computational unit in this method can be equivalent to an operator. This computational unit can be an operator in an AI network or a component of an operator. The combination information in this method can include a combination pattern of at least two computational units. For example, if the data length of the first data shape in a certain dimension is 11, the combination pattern of at least two computational units in the combination information can be a computational unit with a data length of 5 + a computational unit with a data length of 5 + a computational unit with a data length of 5; or it can be a computational unit with a data length of 5 + a computational unit with a data length of 5 + a computational unit with a data length of 1.

[0013] The parameter data in this method can be stored in the cache in the form of a parameter table.

[0014] In one possible implementation, the at least two computing units include the same computing unit; or different computing units; or the same computing unit and different computing units.

[0015] Among them, the second data shape of the same computing unit has the same data length in each dimension; the second data shape of different computing units has different data lengths in at least one dimension.

[0016] In other words, in this implementation, the same and different computing units can be determined by whether the data length is the same in each dimension.

[0017] In one possible implementation, both computing units are computing units of the AI ​​network.

[0018] In other words, in this implementation, the calculation of the first target data of the first data shape that the AI ​​network supports processing can be achieved by calling at least two computing units of the AI ​​network.

[0019] In this approach, at least two computational units can be computational units of an AI network, or computational units of other networks besides an AI network. These AI networks and other networks can be used to implement different functions, such as object detection, image classification, audio processing, and natural language processing.

[0020] For AI networks and other networks that implement different functions, they can include the same computational units, such as both AI networks and other networks including convolutional computation units; or they can include different computational units, with the AI ​​network not including convolutional computation units, while other networks include convolutional operator units. For example, if the AI ​​network does not include convolutional computation units, but other networks do, when the AI ​​network needs to use convolutional computation units, it can use the convolutional operator units included in the other networks.

[0021] In one possible implementation, the combination information includes the combination pattern of the at least two computing units;

[0022] The second data shape of each computing unit, after being combined according to the combination pattern, has a data length in any dimension that is greater than or equal to the data length of the first data shape in the same dimension.

[0023] In other words, in this implementation, the relationship between the second data shape and the first data shape is that the data length of the second data shape of each computing unit in any dimension after being combined according to a certain combination pattern is greater than or equal to the data length of the first data shape in the same dimension.

[0024] In one possible implementation, the parameter data further includes identification information for a specified computing unit;

[0025] The designated computing unit refers to the computing unit among the at least two computing units that needs to process data with a third data shape, wherein the data length of the third data shape in at least one dimension is less than the data length of the second data shape that the designated computing unit supports processing in the same dimension.

[0026] In other words, in this implementation, for a specific calculation unit that needs to process data with a third data shape, identification information can be added to the parameter data for that specific calculation unit so that the specified calculation unit can be called later to perform calculations on data with a third data shape, thereby improving the accuracy of operator calculations.

[0027] In one possible implementation, the parameter data further includes a specified processing method for the data having a third data shape by the specified calculation unit.

[0028] In other words, in this implementation, for a specific calculation unit that needs to process data of the third data shape, a specific processing method can be added to the parameter data for that specific calculation unit, so that the specific calculation unit can be called later to perform calculations on the data of the third data shape using the specified processing method.

[0029] In one possible implementation, the specified processing method includes:

[0030] Discard invalid data, where invalid data is data in the second data shape supported by the designated computing unit that is other than data having the third data shape; or

[0031] Data overlap refers to overlapping the invalid data with data that another computing unit needs to process.

[0032] In other words, in this implementation, the specified processing method can be either discarding invalid data or data overlap, so that subsequent calculations can be performed on the data of the third data shape according to the specified processing method. This enriches the implementation methods of operator calculation and improves the reliability of operator calculation.

[0033] In one possible implementation, the parameter data also includes the specified calculation unit supporting a specified range of variation of the third data shape in each dimension.

[0034] In other words, in this implementation, since the second data shape that the specified computing unit supports processing is fixed, the third data shape that the specified computing unit needs to process can change, but its change has a certain range. Therefore, the specified range of change of the third data shape in each dimension can be added to the parameter data so that the same computing unit can support the change of data shape within a certain range.

[0035] In one possible implementation, the specified range of variation is the data length of the second data shape supported by the specified computing unit in each dimension; or the length of a specified portion of the data length of the second data shape in each dimension.

[0036] In other words, in this implementation method, different ranges of variation can be selected according to the actual situation. If the data length in each dimension is relatively small, it can vary within the entire data length. For example, if the data length is 16, it can vary within the range of 0 to 16. If the data length in each dimension is relatively large, it can vary within a small range at the end of the data length. For example, if the data length is 100, it can support variations within the range of 90 to 100. This can ensure the efficiency of operator calculation and avoid a large number of redundant calculations.

[0037] In one possible implementation, the parameter data includes graded parameter data, which is used to support data shapes with a specified range of variation.

[0038] In other words, in this implementation, the parameter data of different first data shapes can be the same, i.e., segmented parameter data. This way, it is not necessary for each different data shape to correspond to different parameter data, thereby effectively reducing the number of parameter data in the cache and avoiding resource waste.

[0039] In one possible implementation, the invocation of the at least two computing units to perform calculations on the first target data having the first data shape includes:

[0040] Obtain the at least two computing units from the computing unit operator library;

[0041] The first target data having the first data shape is calculated by the at least two computing units.

[0042] In other words, in this implementation, the computation unit operator library can include many pre-compiled computation units. When performing operator calculations, these units can be directly obtained from the computation unit operator library, thereby improving the efficiency of operator calculations and increasing the startup speed of the AI ​​network.

[0043] The computational unit operator library includes computational units that can be used to perform different operations, such as convolution, addition, and matrix multiplication. These computational units that perform different operations can be used by multiple AI networks to achieve different functions, such as object detection, image classification, audio processing, and natural language processing.

[0044] In one possible implementation, the invocation of the at least two computing units to perform calculations on the first target data having the first data shape includes:

[0045] For any computing unit, determine the target position of the second target data that needs to be processed in the first target data;

[0046] According to the target location, the second target data that the computing unit needs to process is obtained from the memory space storing the first target data;

[0047] The second target data is calculated using any of the aforementioned computing units.

[0048] In other words, in this implementation, when performing operator calculations, the target position of the second target data to be processed in the first target data can be determined first, and then the second target data can be obtained from the memory space according to the target position. The calculation of the second target data is then completed by the calculation unit, thereby improving the reliability of operator calculations.

[0049] In this approach, the memory space refers to the storage space in memory used to store data, and its address is one-dimensional. When retrieving the second target data from the memory space, since the second target data may be multi-dimensional, a skip-read / skip-write method is needed to retrieve the second target data from the memory space. After the calculation is completed, the skip-read / skip-write method is used again to save the calculated output data to the memory space.

[0050] It is worth noting that when determining the target position of the second target data to be processed within the first target data, if the parameter data includes the position information of each second data shape within the first data shape, the target position of the second target data within the first target data can be determined based on this position information.

[0051] In one possible implementation, the target location includes: the dimensions in which the second target data is located; and, for any given dimension, the offset and data length of the second target data in that dimension.

[0052] In other words, in this implementation method, since the second target data may be multidimensional, the target location needs to include each dimension where the second target data is located, as well as the offset and data length of the second target data in any dimension, thereby improving the accuracy and efficiency of target data acquisition.

[0053] In one possible implementation, the at least two computational units belong to different types of operators.

[0054] In other words, in this implementation, at least two computational units in the parameter data can belong to the same type of operator, i.e., perform the same function; or they can belong to different types of operators, i.e., perform different functions, such as: convolution operator, add (addition) operator, matmul (matrix multiplication) operator, etc.

[0055] In this approach, the different types of operators can refer to the individual operators cascaded into a fusion operator. A fusion operator is a cascaded set of different types of operators that are combined into a single operator for computation. In this case, at least two computational units in the parameter data can be computational units of these different types of operators, such as computational units for the conv operator, relu operator, abs operator, and exp operator. Thus, during operator computation, the fusion operator can be computed by calling these computational units of different types of operators in the parameter data, avoiding the need to compute one type of operator before calling another type, thereby improving the computational efficiency of the fusion operator.

[0056] In one possible implementation, the computational unit is a pre-compiled operator.

[0057] In other words, in this implementation, the computational unit can be equivalent to an operator, and it is a pre-compiled operator. For example, the computational unit operator library can include many pre-compiled computational units. When performing operator calculations, these units can be directly retrieved from the computational unit operator library, thereby improving the efficiency of operator calculations and increasing the startup speed of the AI ​​network.

[0058] In this approach, the pre-compiled operators can be either static computation unit binary packages compiled by the compilation host and distributed, requiring only the import of these packages by all execution hosts, or multiple computation units pre-compiled by the execution hosts and cached in a cache. During operator computation, these pre-compiled units can be directly retrieved from the cache, improving computation efficiency and AI network startup speed. It's worth noting that "static" in "static computation unit" refers to the fact that the data shape processed by the computation unit remains fixed, allowing pre-compiled units to be used directly for operator computation without recompilation.

[0059] Secondly, embodiments of this application provide an operator computing device, the device comprising:

[0060] The acquisition module is configured to acquire parameter data of a first data shape of the AI ​​network. The first data shape is the data length in each dimension that the AI ​​network supports processing. The parameter data includes combination information of at least two computing units. Each computing unit supports processing data with a second data shape. The data length of the second data shape of each computing unit in any dimension after being combined according to the combination information is greater than or equal to the data length of the first data shape in the same dimension.

[0061] The calculation module is configured to invoke the at least two calculation units to perform calculations on the first target data having the first data shape.

[0062] In one possible implementation, the at least two computing units include the same computing unit; or different computing units; or the same computing unit and different computing units.

[0063] Among them, the second data shape of the same computing unit has the same data length in each dimension; the second data shape of different computing units has different data lengths in at least one dimension.

[0064] In one possible implementation, both computing units are computing units of the AI ​​network.

[0065] In one possible implementation, the combination information includes the combination pattern of the at least two computing units;

[0066] The second data shape of each computing unit, after being combined according to the combination pattern, has a data length in any dimension that is greater than or equal to the data length of the first data shape in the same dimension.

[0067] In one possible implementation, the parameter data also includes identification information for a specific computing unit;

[0068] The designated computing unit refers to the computing unit among the at least two computing units that needs to process data with a third data shape, wherein the data length of the third data shape in at least one dimension is less than the data length of the second data shape that the designated computing unit supports processing in the same dimension.

[0069] In one possible implementation, the parameter data further includes a specified processing method for the data having a third data shape by the specified calculation unit.

[0070] In one possible implementation, the specified processing method includes:

[0071] Discard invalid data, where invalid data is data in the second data shape supported by the designated computing unit that is other than data having the third data shape; or

[0072] Data overlap refers to overlapping the invalid data with data that another computing unit needs to process.

[0073] In one possible implementation, the parameter data also includes the specified calculation unit supporting a specified range of variation of the third data shape in each dimension.

[0074] In one possible implementation, the specified range of variation is the data length of the second data shape supported by the specified computing unit in each dimension; or the length of a specified portion of the data length of the second data shape in each dimension.

[0075] In one possible implementation, the parameter data includes graded parameter data, which is used to support data shapes with a specified range of variation.

[0076] In one possible implementation, the computing module includes:

[0077] The first acquisition submodule is configured to acquire the at least two computing units from the computing unit operator library;

[0078] The first calculation submodule is configured to perform calculations on the first target data having the first data shape using the at least two calculation units.

[0079] In one possible implementation, the computing module includes:

[0080] The determination submodule is configured to determine, for any computing unit, the target position of the second target data to be processed in the first target data;

[0081] The second acquisition submodule is configured to acquire the second target data that the computing unit needs to process from the memory space storing the first target data according to the target location;

[0082] The second calculation submodule is configured to perform calculations on the second target data through any of the calculation units.

[0083] In one possible implementation, the target location includes: the dimensions in which the second target data is located; and, for any given dimension, the offset and data length of the second target data in that dimension.

[0084] In one possible implementation, the at least two computational units belong to different types of operators.

[0085] In one possible implementation, the computational unit is a pre-compiled operator.

[0086] Thirdly, embodiments of this application provide an operator computing device, including:

[0087] At least one memory for storing programs;

[0088] At least one processor is configured to execute a program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to perform the method of the first aspect.

[0089] Fourthly, embodiments of this application provide an operator computing device, which includes the apparatus provided in the second or third aspect.

[0090] Fifthly, embodiments of this application provide an operator computing system, including the operator computing device and operator compilation device provided in the fourth aspect;

[0091] The operator computing device includes the apparatus provided in the second or third aspect;

[0092] The operator compilation device is used to compile publishable computational unit packages;

[0093] The operator computing device is used to import the computing unit package.

[0094] In a sixth aspect, embodiments of this application provide a computer storage medium storing instructions that, when executed on a computer, cause the computer to perform the method provided in the first aspect.

[0095] In a seventh aspect, embodiments of this application provide a computer program product containing instructions that, when executed on a computer, cause the computer to perform the method provided in the first aspect.

[0096] Eighthly, embodiments of this application provide a chip including at least one processor and an interface;

[0097] An interface is used to provide program instructions or data to at least one processor;

[0098] At least one processor is used to execute program line instructions to implement the method provided in the first aspect.

[0099] This application discloses an operator calculation method, apparatus, device, and system. By acquiring parameter data of a first data shape of an AI network, where the first data shape is the data length in each dimension that the AI ​​network supports processing, the parameter data includes combination information of at least two calculation units. Each calculation unit supports processing data with a second data shape, and the data length of the second data shape of each calculation unit in any dimension after being combined according to the combination information is greater than or equal to the data length of the first data shape in the same dimension. At least two calculation units are invoked to calculate the first target data with the first data shape. In this way, by combining at least two calculation units, the changes in data shape within any range can be supported, thereby realizing the AI ​​dynamic shape operator function and improving the startup speed of the AI ​​network. Attached Figure Description

[0100] Figure 1 This is a schematic diagram of the main framework of artificial intelligence;

[0101] Figure 2 This is a schematic diagram of a system architecture for operator computation.

[0102] Figure 3 This is a schematic diagram of data shape changes;

[0103] Figure 4 It is a schematic diagram of the operator calculation process;

[0104] Figure 5 This is a schematic diagram of a system architecture for operator computation.

[0105] Figure 6 This is a component structure diagram of a terminal device;

[0106] Figure 7 This is a hardware structure diagram of an AI chip;

[0107] Figure 8 This is a schematic diagram of a skip-read / skip-write scenario used in the operator calculation process;

[0108] Figure 9 This is a diagram illustrating a skip-read / skip-write support method.

[0109] Figure 10This is a schematic diagram of a tail data processing method;

[0110] Figure 11 This is a schematic diagram of a parameter table structure;

[0111] Figure 12 This is a schematic diagram illustrating an application scenario for a tiered data table.

[0112] Figure 13 This is a schematic diagram of a fusion operator;

[0113] Figure 14 This is a schematic diagram of the operator calculation process;

[0114] Figure 15 This is a schematic diagram of the operator calculation process;

[0115] Figure 16 This is a schematic diagram of the operator calculation process;

[0116] Figure 17 This is a flowchart illustrating an operator calculation method provided in an embodiment of this application;

[0117] Figure 18 This is a schematic diagram of the structure of an operator computing device provided in an embodiment of this application;

[0118] Figure 19 This is a schematic diagram of the structure of an operator computing device provided in an embodiment of this application;

[0119] Figure 20 This is a schematic diagram of the structure of a chip provided in an embodiment of this application. Detailed Implementation

[0120] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be described below with reference to the accompanying drawings.

[0121] In the description of the embodiments of this application, the words "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the words "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a specific manner.

[0122] In the description of the embodiments in this application, the term "and / or" is merely a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, B existing alone, and A and B existing simultaneously. Furthermore, unless otherwise stated, the term "multiple" means two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals.

[0123] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and their variations all mean "including but not limited to," unless otherwise specifically emphasized.

[0124] Figure 1 This is a schematic diagram of the main framework of artificial intelligence, which describes the overall workflow of an artificial intelligence system and is applicable to general artificial intelligence field needs.

[0125] The above-mentioned artificial intelligence framework will be elaborated from two dimensions: "intelligent information chain" (horizontal axis) and "IT value chain" (vertical axis).

[0126] The "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it could be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. In this process, data undergoes a condensation process of "data—information—knowledge—wisdom."

[0127] The "IT value chain" reflects the value that artificial intelligence brings to the information technology industry, from the underlying infrastructure of human intelligence, information (provided and processed by technology) to the industrial ecosystem of systems.

[0128] (1) Infrastructure:

[0129] Infrastructure provides computing power to support artificial intelligence systems, enabling communication with the external world and providing support through a basic platform. This communication occurs through sensors; computing power is provided by intelligent chips (hardware acceleration chips such as CPUs, NPUs, GPUs, ASICs, and FPGAs); and the basic platform includes distributed computing frameworks and related platform guarantees and support, which may include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to acquire data, and this data is provided to intelligent chips in the distributed computing system provided by the basic platform for computation.

[0130] (2) Data

[0131] The data at the next layer of infrastructure is used to represent the data sources in the field of artificial intelligence. The data involves graphics, images, voice, text, and IoT data from traditional devices, including business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0132] (3) Data processing

[0133] Data processing typically includes methods such as data training, machine learning, deep learning, search, reasoning, and decision-making.

[0134] Among them, machine learning and deep learning can perform intelligent information modeling, extraction, preprocessing, and training on data, including symbolization and formalization.

[0135] Reasoning refers to the process in which, in a computer or intelligent system, the machine thinks and solves problems by simulating human intelligent reasoning, based on reasoning control strategies and using formalized information. Typical functions include search and matching.

[0136] Decision-making refers to the process of making decisions based on intelligent information after reasoning, and it typically provides functions such as classification, sorting, and prediction.

[0137] (4) General ability

[0138] After the data processing mentioned above, the results of the data processing can be used to form some general capabilities, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0139] (5) Smart Products and Industry Applications

[0140] Intelligent products and industry applications refer to products and applications of artificial intelligence systems in various fields. They encapsulate overall artificial intelligence solutions, productize intelligent information decision-making, and realize practical applications. Their application areas mainly include: intelligent manufacturing, intelligent transportation, smart home, intelligent healthcare, intelligent security, autonomous driving, safe city, and intelligent terminals.

[0141] It should be noted that the operator calculation involved in this application is located in the data processing stage in (3) above.

[0142] Figure 2 This is a schematic diagram of a system architecture for operator computation, such as... Figure 2As shown, in the AI ​​network initialization phase 4012, the machine learning platform 4011 can parse all the AI ​​operators in the AI ​​network and the data shapes that each AI operator needs to support processing, and use the AI ​​compiler 4018 to complete the operator compilation. In the execution phase of the AI ​​network (i.e., the runtime engine 4014), it can call the operators 4016 and execute the operator calculations (i.e., the execution module 4017). Here, an AI operator can refer to a unit module in the AI ​​network that implements a specific calculation. For example, convolution operators, add operators, matmul (matrix multiplication) operators, etc.

[0143] It is evident that the more complex the AI ​​network, the more types of AI operators it contains. Even for the same type of AI operator, the data shapes it needs to support are becoming increasingly diverse. If an AI operator needs to be recompiled for each different data shape, the compilation process becomes increasingly time-consuming and reduces the startup speed of the AI ​​network.

[0144] It should be noted that the data shape referred to in this application refers to the data length of the data calculated by the operator in each dimension. This data shape can vary in one dimension or simultaneously in multiple dimensions. For example... Figure 3 As shown, the graphic data can change in one dimension (length), or it can change in both length and width.

[0145] To address the aforementioned technical issues, this application provides an operator calculation method, apparatus, device, and system that implements AI dynamic shape operator functionality by combining binary static calculation units. This enables support for changes in data shape within any range and improves the startup speed of AI networks.

[0146] It should be noted that the term "static" in the static computing unit involved in this application refers to the fact that the data shape supported by the computing unit is fixed and unchanging. In this way, the pre-compiled static computing unit can be directly used for operator calculation without recompiling.

[0147] The parameter table involved in this application refers to parameter data in tabular form that describes the first data shape supported by the AI ​​network. The parameter data includes combination information of at least two computing units, and each computing unit supports processing data with a second data shape.

[0148] The following is a description through specific embodiments.

[0149] Figure 4 This is a schematic diagram of an operator calculation process. For example... Figure 4As shown, this operator computation can be used in AI networks, which contain many operators. Examples include convolution operators, add operators, and matmul (matrix multiplication) operators. During AI network initialization, the required operator types are obtained by analyzing the operators and simplifying the network. For AI networks where the data shape is variable during execution, some operators need to support processing data with variable shapes, including... Figure 4 The left side of the diagram represents two scenarios: "data shape remains unchanged during execution" and the right side represents "data shape changes during execution." For AI networks where the data shape remains unchanged during execution, only [the following is included]... Figure 4 The scenario on the left side of the middle section is "data shape remains unchanged during execution". Figure 4 The static computation units can be stored in the static computation unit operator library in advance, and can be retrieved from the static computation unit operator library when the static computation unit is called to complete the calculation. Figure 4 The meanings of each part are as follows:

[0150] Static computation unit: An operator unit that performs calculations only on data with a fixed shape. This operator unit is also equivalent to an operator. Each type of operator can contain several optimized static computation units with different data shapes. Different types of operators can perform different functions, such as: convolution operator, add (addition) operator, matmul (matrix multiplication) operator, etc.

[0151] Parameter table: A data structure used to describe the combination pattern of static computing units. Each static computing unit completes the calculation of a block of data. By combining multiple static computing units according to the parameters in the parameter table, the calculation of all data can be completed.

[0152] AI network initialization: Analyze the initialization operation stage, such as the type of AI network operator.

[0153] The initial execution of the AI ​​network involves invoking operators and completing computations. Operator computation can be accomplished through a combination of parameter tables and static computation units, with the parameter table serving as an input parameter for the static computation unit.

[0154] For scenarios where the data shape remains unchanged during execution: For operators with unchanged data shape, a parameter table can be generated and stored in a cache during AI network initialization, and then retrieved directly from the cache during AI network execution.

[0155] For scenarios where the data shape is variable during execution: For operators with variable data shape, a parameter table can be generated during the execution of the AI ​​network, and then the static computing unit can be called to complete the calculation based on the parameter table. If a caching mechanism is used, the parameter table can be retrieved from the cache when it is used again, without having to regenerate the parameter table every time it is used.

[0156] Figure 5This is a schematic diagram of a system architecture for operator computation. The product implementation of this application consists of program code included in an AI compiler and machine learning / deep learning platform software, deployed on host hardware. Figure 5 Taking the application scenario shown as an example, the program code of this application exists inside the static computing unit compilation module of the AI ​​compiler, the initialization module of the platform software, and the runtime engine. During compilation, the program code of this application runs in the CPU of the compilation host; during runtime, the static computing unit 4016 of this application runs in the AI ​​chip of the execution host, which can be equipped with a binary static computing unit and the software program for the operator calculation process provided by this application. Figure 5 The implementation of this application in the host AI compiler and platform software is shown, where the dashed boxes 4013, 4015, 4016, 4017, and 4019 represent modules newly added to the existing platform software. Within the initialization module 4012, this application designs a combinatorial algorithm module 4013; within the runtime engine 4013, the execution module 4017 can complete operator calculations based on the parameter table 4015 and the invoked static computation unit 4016; the AI ​​compiler 4018 includes a static computation unit compilation module 4019, which, after compiling the static computation units, yields a static computation unit binary package 4020.

[0157] In addition, the above Figure 5 This illustrates a typical application scenario for the release of static computing unit (PC) binary packages. In this scenario, the compilation host 4002 and the execution host 4001 are separate. Before releasing the software version, a releasable PC binary package 4020 is compiled on the compilation host 4002, and all execution hosts 4001 only need to import the static computing unit binary package 4020.

[0158] However, in another application scenario, the machine learning platform 4011 includes an AI compiler 4018, and the static computing unit is compiled on the execution host 4001. That is, the functionality of the compilation host 4002 is implemented on the execution host 4001. In this application scenario, the static computing unit 4016 is compiled during the initialization of 4012.

[0159] Figure 6 This is a component structure diagram of a terminal device. For example... Figure 6As shown, a binary-distributed static computing unit is used on the terminal device to provide AI network execution capabilities to all apps on the terminal device through the general interface NNAPI (Neural Networks Application Programming Interface) 4011. The static computing unit 4016 of this application uses the binary-distributed operator package 4020, which does not require recompilation. During initialization 4012, only the combination algorithm 4013 needs to be called to generate the parameter table 4015 corresponding to the data shape of the operator. This embodiment can minimize the startup time of the AI ​​network when the APP (application) is opened, greatly improving the user experience. Among them, the static computing unit 4016 of this application can run in the AI ​​chip of the terminal device, and the AI ​​chip can be equipped with the binary static computing unit and the software program of the operator calculation process provided in this application.

[0160] It's worth noting that using the released static computing unit binary package on terminal devices can decouple the data shape information from the operator code, reducing the difficulty of operator development and optimization. For example, in application scenarios such as mobile phones, tablets, and smart TVs, using the released static computing unit binary package can significantly improve the initialization speed of AI networks in apps.

[0161] Figure 7 This is a hardware structure diagram of an AI chip. This AI chip can be equipped with a binary static computing unit and the software program for the operator computation process provided in this application. The Neural Network Processor (NPU) 50 is mounted as a coprocessor on the host CPU, and tasks are allocated by the host CPU. The core of the NPU is the arithmetic circuit 50, which, through the controller 504, controls the arithmetic circuit 503 to retrieve matrix data from memory and perform multiplication operations.

[0162] In some implementations, the arithmetic circuit 503 internally includes multiple processing engines (PEs). In some implementations, the arithmetic circuit 503 is a two-dimensional pulsating array. The arithmetic circuit 503 can also be a one-dimensional pulsating array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 503 is a general-purpose matrix processor.

[0163] For example, suppose we have an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from the weight memory 502 and caches it in each PE of the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from the input memory 501 and performs matrix operations with matrix B. The partial result or the final result of the obtained matrix is ​​stored in the accumulator 508.

[0164] Unified memory 506 is used to store input and output data. Weight data is directly accessed via the Direct Memory Access Controller (DMAC) 505 and moved to weight memory 502. Input data is also moved to unified memory 506 via DMAC.

[0165] BIU stands for Bus Interface Unit 510, which is used for interaction between the AXI bus and the DMAC and the instruction fetch buffer 509.

[0166] The Bus Interface Unit (BIU) 510 is used by the instruction fetch memory 509 to fetch instructions from external memory, and also by the memory access controller 505 to fetch the original data of the input matrix A or the weight matrix B from external memory.

[0167] The DMAC is mainly used to move input data from external memory DDR to unified memory 506, or to weight data to weight memory 502, or to input data to input memory 501.

[0168] The vector computation unit has over 507 processing units that further process the output of the computation circuits as needed, such as vector multiplication, vector addition, exponential operations, logarithmic operations, size comparisons, etc. It is primarily used for computation in non-convolutional / FC layers of neural networks, such as pooling, batch normalization, and local response normalization.

[0169] In some implementations, the vector computation unit 507 can store the processed output vector into a unified buffer 506. For example, the vector computation unit 507 can apply a nonlinear function to the output of the arithmetic circuit 503, such as a vector of accumulated values, to generate activation values. In some implementations, the vector computation unit 507 generates normalized values, merged values, or both. In some implementations, the processed output vector can be used as activation input to the arithmetic circuit 503, for example, for use in subsequent layers of a neural network.

[0170] The instruction fetch buffer 509 connected to the controller 504 is used to store the instructions used by the controller 504;

[0171] Unified memory 506, input memory 501, weighted memory 502, and instruction fetch memory 509 are all on-chip memories. External memory is proprietary to this NPU hardware architecture.

[0172] Figure 8 This is a schematic diagram illustrating a skip-read / skip-write scenario in operator computation. The static computation unit in this application only performs calculations on data with a fixed shape. Its key implementation lies in the data space, which needs to be segmented to extract the data required for each calculation. This data space refers to the logical space defined by the data shape for storing data; its address is multi-dimensional, but it actually resides in memory. Memory space, on the other hand, can refer to the storage space in memory used to store data; its address is one-dimensional. For example... Figure 8 As shown, the gray area represents the data that a static calculation unit needs to calculate. After expanding the memory space, it can be seen that the gray area is discontinuous. Therefore, a skip-read / skip-write approach can be used to retrieve the data required for each static calculation unit from the memory space, and after the calculation is completed, the output data obtained from the calculation can be saved to the memory space.

[0173] Figure 9 This is a schematic diagram illustrating a skip-read / skip-write support method. One method for supporting skip-read / skip-write in this application is to add an interface, such as the `bind_buffer(axis, stride, offset)` interface. This interface establishes the mapping relationship between the Tensor and buffer of the static computation unit. Figure 8 As shown, a Tensor corresponds to the data that this static computation unit needs to process. Figure 9(The gray area represents the buffer, whose size is consistent with the size of the data stored in the data space.) Here, Axis refers to the dimensions in which the Tensor resides; Offset is the offset of the Tensor in the dimension where data needs to be read; Stride refers to the length of data that the Tensor needs to skip in the dimension where data needs to be read. Therefore, this application can achieve skip-read and skip-write operations in a multi-dimensional data space by mapping multiple dimensions.

[0174] Figure 10 This is a diagram illustrating a tail data processing method. The tail data refers to a segment of data that, when logically divided along one dimension by a static computing unit, has a length smaller than the length the static computing unit can process. For example... Figure 10 As shown, there are four ways to process this tail data:

[0175] Method 1: Discard invalid data, that is, discard the excess data at the end. Advantages: small scalar computation. Disadvantages: many invalid computations.

[0176] Method 2 involves partial data overlap, where the excess data at the tail is shifted forward, resulting in partial data overlap. Advantages: smaller scalar computation; disadvantages: more redundant computation. In other words, the excess data at the tail can overlap with data that another computation unit needs to process. Specifically, when the static computation unit calculating the tail data reads data, it can set its starting position within the data that another computation unit needs to process, thus causing partial data overlap.

[0177] Method 3 supports changes across the entire data range. This means that when the data length in each dimension is relatively small, changes can occur within the entire data length. For example, if the data length is 16, changes can be made within the range of 0 to 16.

[0178] Method 4 supports variations within a specific range of data. This means that when the data length in each dimension is relatively large, variations are only supported within a small range at the end of that data length. For example, if the data length is 100, variations within the range of 90 to 100 are supported.

[0179] Methods 3 and 4 above both handle data shape changes. Their advantages are fewer redundant calculations, but their disadvantages include more scalar calculations.

[0180] Figure 11 This is a schematic diagram of a parameter table structure. A parameter table is a form of parameter data used to describe the shape of the data that an AI network can support processing. For example... Figure 11As shown, the left side illustrates a logical segmentation method for the target data shape supported by the AI ​​network (it can also be segmented into data shapes of different sizes). Each data shape obtained from the logical segmentation corresponds to one static computation unit call. The right side shows the data structure of the parameter table. The parameter table is a data structure used to describe the combination pattern of static computation units. Each item in the table corresponds to one static computation unit call and the position of the data to be calculated by the called static computation unit in the target data. The data parameters in the table mainly include:

[0181] 1) The ID of the static computing unit to be invoked;

[0182] 2) The length of the target data in each dimension, for example: Figure 11 In the data length section, 0 represents the length of the target data in the width dimension, and 1 represents the length of the target data in the length dimension.

[0183] 3) The offset of the data to be calculated in each dimension by the static calculation unit to be invoked, for example: Figure 11 In the text, offset 0 represents the offset of the target data in the width dimension, and offset 1 represents the offset of the target data in the length dimension.

[0184] 4) The length of the data to be calculated in each dimension for the static calculation unit to be called.

[0185] In addition, the parameter table has a common parameter area to store the same parameters in each table item, such as the data length of the target data in each dimension.

[0186] The parameter table is generated by the combinatorial algorithm. For example, in two dimensions, the combinatorial algorithm is equivalent to assembling rectangular blocks; in three dimensions, it is equivalent to assembling cubes.

[0187] The parameter table can be divided into data tables. This is because the static calculation unit can be of variable length when processing tail data. For example, such as... Figure 12 The three examples shown all support data shapes with a certain range of variation, so it's unnecessary to generate a parameter table for each data shape. Using a tiered parameter table format can effectively reduce the number of parameter tables in the cache. Figure 12 The range of change on the left side refers to global changes. For example, if the data length in a certain dimension is 16, the global range of change can be 0 to 16. The range of change in the middle refers to partial changes. For example, if the data length in a certain dimension is 100, the partial range of change can be 90 to 100. The range of change on the right side refers to changes only in the tail data.

[0188] It's worth noting that the parameter table corresponding to the data shapes that the AI ​​network needs to support is automatically generated and requires no user configuration. The features of this parameter table can include: 1) a data structure that describes the order in which static computation units are called; 2) a description of the position of the data block computed by each static computation unit within the original data (i.e., the position of the data block with the second data shape corresponding to each static computation unit within the data block with the first data shape); and 3) the parameter table can describe the combination mode of the static computation units, controlling the static computation units to complete the computation of all data. The static computation units can be binary static computation units. That is, the static computation units can be compiled into binary files before the version is released and provided to users in binary file form upon release.

[0189] Furthermore, by combining binary static computation units, it can support changes in the shape of data within any range. The features of this approach include: 1) only one block of data can be computed at a time; 2) it supports extracting a block of data from the original data space using a skip-read method, and writing it to the corresponding position in the output data space using a skip-write method after computation; 3) it supports data-driven computation, including specifying which static computation unit to call and which block of data to compute, all of which can be imported as data through a parameter table.

[0190] Figure 13 This is a schematic diagram of a fusion operator, which refers to cascading different types of operators that are combined into a single operator for computation. The static computation unit 4016 can contain static computation units of different types of operators, distinguished by their IDs during invocation. Then, the parameter table 4015 describes the invocation order and related parameters of the static computation units of different types of operators, thus completing the computation of the fusion operator. For example... Figure 13 As shown, the static computation unit 4016 includes static computation units for the conv operator, the relu operator, the abs operator, and the exp operator, which are distinguished by ID when called. The parameter table 4015 describes the calling order and related parameters of the static computation units for the conv, relu, abs, and exp operators. The related parameters may include the size of the second data shape that each static computation unit supports processing; they may also include the position information of the second data shape data supported by each static computation unit within the data having the first data shape.

[0191] Figure 14 This is a schematic diagram of the operator computation process. When an operator is invoked during the execution of an AI network, such as... Figure 14 As shown, the parameter table is used as one of the input parameters of the static calculation unit. The operator will call the static calculation unit in sequence according to the contents of the parameter table to complete the operator calculation.

[0192] Figure 15 It is a schematic diagram of the operator calculation process, such as Figure 15 As shown, the calculation process on the left is illustrated. Figure 1 This refers to the operator computation process in a three-dimensional single-input scenario; the computation process is illustrated on the right. Figure 2 This refers to two operator inputs: a two-dimensional input data and a one-dimensional input data.

[0193] Figure 16 It is a schematic diagram of the operator calculation process, such as Figure 16 As shown, when iterating through the data shapes that each AI operator needs to support, the system first checks if a supported parameter table exists in the cache. If it does, the cached parameter table is used; otherwise, a simplified combination algorithm is used to generate a parameter table and store it in the cache. Simultaneously, a thread is started to call an optimized combination algorithm to generate a parameter table and update the cache. This allows for the fastest possible network initialization when the AI ​​network is first started, and subsequent times when the AI ​​network is opened, the optimized parameter table can be used, thus improving the user experience. The simplified combination algorithm refers to using only the same static computation units, resulting in the shortest combination time; the optimized combination algorithm refers to using different static computation units for optimal combination, resulting in a longer combination time.

[0194] Next, please refer to Figure 17 , Figure 17 This is a flowchart illustrating an operator computation method provided in an embodiment of this application. This operator computation method can be used in AI networks. Figure 17 As shown, the operator calculation method may include the following steps:

[0195] S171. Obtain parameter data of the first data shape of the AI ​​network. The first data shape is the data length in each dimension that the AI ​​network supports processing. The parameter data includes combination information of at least two computing units. The data that each computing unit supports processing is data with a second data shape. The data length of the second data shape of each computing unit in any dimension after being combined according to the combination information is greater than or equal to the data length of the first data shape in the same dimension.

[0196] In some embodiments, at least two computational units in the parameter data may include the same computational unit; or different computational units; or the same computational unit and different computational units; wherein, the second data shape of the same computational unit has the same data length in each dimension; and the second data shape of different computational units has different data lengths in at least one dimension.

[0197] At least two computational units in the parameter data can both be computational units of the AI ​​network.

[0198] The combination information of at least two computational units in the parameter data may include a combination pattern, such that the data length of the second data shape of each computational unit after being combined according to the combination pattern is greater than or equal to the data length of the first data shape in the same dimension.

[0199] The parameter data may also include identification information for a specified computing unit; wherein, the specified computing unit refers to the computing unit among the at least two computing units that needs to process data with a third data shape, and the data length of the third data shape in at least one dimension is less than the data length of the second data shape that the specified computing unit supports processing in the same dimension. For example, the data of the third data shape may be... Figure 10 The tail data involved.

[0200] The parameter data may also include a specified processing method for the computing unit on data having a third data shape. This specified processing method may include: discarding invalid data, where the invalid data is data in the second data shape that the specified computing unit supports processing, excluding data with the third data shape; or data overlap, where the invalid data is overlapped with data that another computing unit needs to process. For example, discarding invalid data could be… Figure 10 Method 1 involved; data overlap can be Figure 10 Method 2 involved in the process.

[0201] The parameter data may also include a specified range of variation in each dimension for the third data shape supported by the specified computing unit. The specified range of variation can be the data length of the second data shape supported by the specified computing unit in each dimension; or a specified portion of the data length of the second data shape in each dimension. For example, the specified range of variation could be... Figure 10 Methods 3 and 4 are involved.

[0202] The parameter data includes graded parameter data, which is used to support data shapes within a specified range of variation. For example, such as... Figure 12 The three examples shown can all support data shapes with a certain range of variation, so it is not necessary to generate a parameter table for each data shape. If a parameter table is used in the form of a tiered parameter table, the number of parameter tables in the cache can be effectively reduced.

[0203] At least two computational units in the parameter data can belong to different types of operators. This allows the computation of the fusion operator to be completed by calling these computational units of different operator types within the parameter data. This avoids having to call another type of operator after one type has been computed, thus improving the computational efficiency of the fusion operator. For example, such as... Figure 13As shown, the static computation unit 4016 includes static computation units for the conv operator, the relu operator, the abs operator, and the exp operator, which are distinguished by ID when called; the parameter table 4015 describes the calling order and related parameters of the static computation units for the conv operator, the relu operator, the abs operator, and the exp operator.

[0204] S172. Call at least two computing units to perform calculations on the first target data having a first data shape.

[0205] In some embodiments, at least two computing units may be obtained from a library of computing unit operators; calculations may be performed on first target data having a first data shape using at least two computing units. For example, such as... Figure 5 As shown, the execution host 4001 can import the static computing unit binary package 4020 published by the compilation host 4002, so that at least two computing units can be obtained from the static computing unit binary package 4020, and the first target data having a first data shape can be calculated by the at least two computing units.

[0206] The target position of the second target data to be processed in any computing unit within the first target data can be determined; the second target data to be processed by any computing unit can be retrieved from the memory space storing the first target data according to the target position; and the second target data can be calculated by any computing unit. For example, such as... Figure 11 As shown, the left side illustrates a logical segmentation method for the target data shape supported by the AI ​​network (it can also be segmented into data shapes of different sizes). Each data shape obtained from the logical segmentation corresponds to one static computation unit call. The right side shows the data structure of the parameter table. The parameter table is a data structure used to describe the combination pattern of static computation units. Each item in the table corresponds to one static computation unit call and the position of the data to be calculated by the called static computation unit in the target data. This allows the data to be calculated to be obtained based on the position of the data to be calculated by the called static computation unit in the target data, and the called static computation unit to perform the calculation on the data to be calculated.

[0207] Therefore, the above scheme can support changes in the shape of data within any range, realize the AI ​​dynamic shape operator function, and improve the startup speed of AI network.

[0208] Based on the methods in the above embodiments, this application also provides an operator computing device, wherein the operator computing device is used in AI networks. Please refer to... Figure 18 , Figure 18This is a schematic diagram of the structure of an operator computing device provided in an embodiment of this application, as shown below. Figure 18 As shown, the operator computing device includes:

[0209] The acquisition module 181 is configured to acquire parameter data of the first data shape of the AI ​​network. The first data shape is the data length in each dimension that the AI ​​network supports processing. The parameter data includes combination information of at least two computing units. The data that each computing unit supports processing is data with a second data shape. The data length of the second data shape of each computing unit in any dimension after being combined according to the combination information is greater than or equal to the data length of the first data shape in the same dimension.

[0210] The calculation module 182 is configured to invoke the at least two calculation units to perform calculations on the first target data having the first data shape.

[0211] In one possible implementation, the at least two computing units include the same computing unit; or different computing units; or the same computing unit and different computing units.

[0212] Among them, the second data shape of the same computing unit has the same data length in each dimension; the second data shape of different computing units has different data lengths in at least one dimension.

[0213] In one possible implementation, the at least two computing units are computing units of an AI network.

[0214] In one possible implementation, the combination information includes the combination pattern of the at least two computing units;

[0215] The second data shape of each computing unit, after being combined according to the combination pattern, has a data length in any dimension that is greater than or equal to the data length of the first data shape in the same dimension.

[0216] In one possible implementation, the parameter data also includes identification information for a specific computing unit;

[0217] The designated computing unit refers to the computing unit among the at least two computing units that needs to process data with a third data shape, wherein the data length of the third data shape in at least one dimension is less than the data length of the second data shape that the designated computing unit supports processing in the same dimension.

[0218] In one possible implementation, the parameter data further includes a specified processing method for the data having a third data shape by the specified calculation unit.

[0219] In one possible implementation, the specified processing method includes:

[0220] Discard invalid data, where invalid data is data in the second data shape supported by the designated computing unit that is other than data having the third data shape; or

[0221] Data overlap refers to overlapping the invalid data with data that another computing unit needs to process.

[0222] In one possible implementation, the parameter data also includes the specified calculation unit supporting a specified range of variation of the third data shape in each dimension.

[0223] In one possible implementation, the specified range of variation is the data length of the second data shape supported by the specified computing unit in each dimension; or the length of a specified portion of the data length of the second data shape in each dimension.

[0224] In one possible implementation, the parameter data includes graded parameter data, which is used to support data shapes with a specified range of variation.

[0225] In one possible implementation, the computing module 182 includes:

[0226] The first acquisition submodule is configured to acquire the at least two computing units from the computing unit operator library;

[0227] The first calculation submodule is configured to perform calculations on the first target data having the first data shape using the at least two calculation units.

[0228] In one possible implementation, the computing module 182 includes:

[0229] The determination submodule is configured to determine, for any computing unit, the target position of the second target data to be processed in the first target data;

[0230] The second acquisition submodule is configured to acquire the second target data that the computing unit needs to process from the memory space storing the first target data according to the target location;

[0231] The second calculation submodule is configured to perform calculations on the second target data through any of the calculation units.

[0232] In one possible implementation, the target location includes: the dimensions in which the second target data is located; and, for any given dimension, the offset and data length of the second target data in that dimension.

[0233] In one possible implementation, the at least two computational units belong to different types of operators.

[0234] In one possible implementation, the computational unit is a pre-compiled operator.

[0235] It should be understood that the above-described device is used to execute the methods in the above embodiments. The implementation principle and technical effect of the corresponding program modules in the device are similar to those described in the above methods. The working process of the device can be referred to the corresponding process in the above methods, and will not be repeated here.

[0236] Based on the methods in the above embodiments, this application also provides an operator computing device. Please refer to... Figure 19 , Figure 19 This is a schematic diagram of the structure of an operator computing device provided in an embodiment of this application. For example... Figure 19 As shown in the embodiments of this application, the operator computing device provided can be used to implement the methods described in the above method embodiments.

[0237] The operator computing device includes at least one processor 1601, which can support the operator computing device in implementing the control method provided in the embodiments of this application.

[0238] The processor 1601 can be a general-purpose processor or a special-purpose processor. For example, the processor 1601 may include a central processing unit (CPU) and / or a baseband processor. The baseband processor can be used to process communication data (e.g., determine the target screen terminal), and the CPU can be used to implement corresponding control and processing functions, execute software programs, and process data from the software programs.

[0239] Furthermore, the operator computing device may also include a transceiver unit 1605 for inputting (receiving) and outputting (transmitting) signals. For example, the transceiver unit 1605 may include a transceiver or an RF chip. The transceiver unit 1605 may also include a communication interface.

[0240] Optionally, the operator computing device may also include an antenna 1606, which can be used to support the transceiver unit 1605 in realizing the transceiver function of the operator computing device.

[0241] Optionally, the operator computing device may include one or more memories 1602 storing a program (which may also be instructions or code) 1604. The program 1604 can be executed by the processor 1601, causing the processor 1601 to perform the methods described in the above method embodiments. Optionally, the memory 1602 may also store data. Optionally, the processor 1601 may also read data stored in the memory 1602 (e.g., pre-stored first feature information), which may be stored at the same storage address as the program 1604, or it may be stored at a different storage address than the program 1604.

[0242] The processor 1601 and memory 1602 can be configured separately or integrated together, for example, integrated on a single board or system on chip (SOC).

[0243] For a detailed description of the operations performed by the operator computing device in the various possible designs described above, please refer to the description in the embodiments of the operator computing method provided in this application, and will not be repeated here.

[0244] Based on the apparatus in the above embodiments, this application also provides an operator computing device, which includes any of the operator computing devices provided in the above embodiments.

[0245] It is understood that in this application embodiment, the operator computing device can be a mobile phone, tablet computer, digital camera, personal digital assistant (PDA), wearable device, smart TV, Huawei Smart Screen, and other terminal devices. Exemplary embodiments of the terminal device include, but are not limited to, terminal devices running iOS, Android, Windows, Harmony OS, or other operating systems. The aforementioned terminal devices can also be other terminal devices, such as laptops with touch-sensitive surfaces (e.g., touch panels). This application embodiment does not specifically limit the type of terminal device. The component structure diagram of the terminal device is shown below. Figure 6 As shown.

[0246] Based on the operator computing device in the above embodiments, this application also provides an operator computing system, an operator computing device, and an operator compilation device; wherein, the operator computing device includes any of the operator computing apparatuses provided in the above embodiments; the operator compilation device is used to compile a publishable static computing unit binary package; the operator computing device is used to import the static computing unit binary package. For example, the operator computing device can be... Figure 5 Execution host 4001 or Figure 6 The terminal device and operator compilation device in the middle can beFigure 5 The compilation host in the middle is 4002.

[0247] Based on the methods described in the above embodiments, this application also provides a chip. Please refer to... Figure 20 , Figure 20 This is a schematic diagram of a chip structure provided in an embodiment of this application. Figure 20 As shown, chip 1900 includes one or more processors 1901 and interface circuitry 1902. Optionally, chip 1900 may also include a bus 1903. Wherein:

[0248] Processor 1901 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed through integrated logic circuits in the hardware of processor 1901 or through software instructions. The processor 1901 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods and steps disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor.

[0249] The interface circuit 1902 can be used to send or receive data, instructions or information. The processor 1901 can use the data, instructions or other information received by the interface circuit 1902 to process the data, instructions or other information, and can send the processed information out through the interface circuit 1902.

[0250] Optionally, the chip may also include memory, which may include read-only memory and random access memory, and provide operation instructions and data to the processor. A portion of the memory may also include non-volatile random access memory (NVRAM).

[0251] Optionally, the memory stores executable software modules or data structures, and the processor can execute corresponding operations by calling the operation instructions stored in the memory (which may be stored in the operating system).

[0252] Optionally, the interface circuit 1902 can be used to output the execution results of the processor 1901.

[0253] It should be noted that the functions of the processor 1901 and the interface circuit 1902 can be implemented through hardware design, software design, or a combination of hardware and software; no restrictions are imposed here.

[0254] It should be understood that each step of the above method embodiments can be completed by hardware logic circuits or software instructions in a processor.

[0255] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0256] The method steps in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0257] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0258] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application.

Claims

1. An operator calculation method, characterized in that, The method is applied to a processor used to deploy an artificial intelligence (AI) network, and the method includes: Obtain parameter data of the first data shape of the artificial intelligence (AI) network. The first data shape is the data length in each dimension that the AI ​​network supports processing. The parameter data includes graded parameter data for supporting data shapes with a specified range of variation. The parameter data includes combination information of at least two computing units. The data that each computing unit supports processing is data with a second data shape. The data length of the second data shape of each computing unit after being combined according to the combination information is greater than or equal to the data length of the first data shape in the same dimension. Based on the parameter data, the at least two computing units are invoked to perform calculations on the first target data having the first data shape.

2. The method according to claim 1, characterized in that, The at least two computing units include the same computing unit; or different computing units; or the same computing unit and different computing units. Among them, the second data shape of the same computing unit has the same data length in each dimension; the second data shape of different computing units has different data lengths in at least one dimension.

3. The method according to claim 1, characterized in that, The at least two computing units are computing units of the AI ​​network.

4. The method according to claim 1, characterized in that, The combination information includes the combination pattern of the at least two computing units; The second data shape of each computing unit, after being combined according to the combination pattern, has a data length in any dimension that is greater than or equal to the data length of the first data shape in the same dimension.

5. The method according to claim 1, characterized in that, The parameter data also includes identification information for a specific computing unit; The designated computing unit refers to the computing unit among the at least two computing units that needs to process data with a third data shape, wherein the data length of the third data shape in at least one dimension is less than the data length of the second data shape that the designated computing unit supports processing in the same dimension.

6. The method according to claim 5, characterized in that, The parameter data also includes the specified processing method of the specified calculation unit for the data with the third data shape.

7. The method according to claim 6, characterized in that, The specified processing method includes: Discard invalid data, where invalid data is data in the second data shape supported by the designated computing unit that is other than data having the third data shape; or Data overlap refers to overlapping the invalid data with data that another computing unit needs to process.

8. The method according to claim 5, characterized in that, The parameter data also includes the specified calculation unit supporting the specified range of variation of the third data shape in each dimension.

9. The method according to claim 8, characterized in that, The specified range of variation is the data length of the second data shape supported by the specified computing unit in each dimension; or the length of a specified portion of the data length of the second data shape in each dimension.

10. The method according to claim 1, characterized in that, The step of calling the at least two computing units to perform calculations on the first target data having the first data shape includes: Obtain the at least two computing units from the computing unit operator library; The first target data having the first data shape is calculated by the at least two computing units.

11. The method according to claim 1, characterized in that, The step of calling the at least two computing units to perform calculations on the first target data having the first data shape includes: For any computing unit, determine the target position of the second target data that needs to be processed in the first target data; According to the target location, the second target data that the computing unit needs to process is obtained from the memory space storing the first target data; The second target data is calculated using any of the aforementioned computing units.

12. The method according to claim 11, characterized in that, The target location includes: each dimension in which the second target data is located; and, for any dimension, the offset and data length of the second target data in that dimension.

13. The method according to claim 1, characterized in that, The at least two computational units belong to different types of operators.

14. The method according to any one of claims 1 to 13, characterized in that, The computational unit is a pre-compiled operator.

15. An operator computing device, characterized in that, The device includes: The acquisition module is configured to acquire parameter data of a first data shape of an artificial intelligence (AI) network. The first data shape is the data length in each dimension that the AI ​​network supports processing. The parameter data includes graded parameter data for supporting data shapes with a specified range of variation. The parameter data includes combination information of at least two computing units. Each computing unit supports processing data with a second data shape. The data length of the second data shape of each computing unit after being combined according to the combination information in any dimension is greater than or equal to the data length of the first data shape in the same dimension. The calculation module is configured to invoke the at least two calculation units based on the parameter data to perform calculations on the first target data having the first data shape.

16. The apparatus according to claim 15, characterized in that, The at least two computing units include the same computing unit; or different computing units; or the same computing unit and different computing units. Among them, the second data shape of the same computing unit has the same data length in each dimension; the second data shape of different computing units has different data lengths in at least one dimension.

17. The apparatus according to claim 15, characterized in that, The at least two computing units are computing units of the AI ​​network.

18. The apparatus according to claim 15, characterized in that, The combination information includes the combination pattern of the at least two computing units; The second data shape of each computing unit, after being combined according to the combination pattern, has a data length in any dimension that is greater than or equal to the data length of the first data shape in the same dimension.

19. The apparatus according to claim 15, characterized in that, The parameter data also includes identification information for a specific computing unit; The designated computing unit refers to the computing unit among the at least two computing units that needs to process data with a third data shape, wherein the data length of the third data shape in at least one dimension is less than the data length of the second data shape that the designated computing unit supports processing in the same dimension.

20. The apparatus according to claim 19, characterized in that, The parameter data also includes the specified processing method of the specified calculation unit for the data with the third data shape.

21. The apparatus according to claim 20, characterized in that, The specified processing method includes: Discard invalid data, where invalid data is data in the second data shape supported by the designated computing unit that is other than data having the third data shape; or Data overlap refers to overlapping the invalid data with data that another computing unit needs to process.

22. The apparatus according to claim 19, characterized in that, The parameter data also includes the specified calculation unit supporting the specified range of variation of the third data shape in each dimension.

23. The apparatus according to claim 22, characterized in that, The specified range of variation is the data length of the second data shape supported by the specified computing unit in each dimension; or the length of a specified portion of the data length of the second data shape in each dimension.

24. The apparatus according to claim 15, characterized in that, The computing module includes: The first acquisition submodule is configured to acquire the at least two computing units from the computing unit operator library; The first calculation submodule is configured to perform calculations on the first target data having the first data shape using the at least two calculation units.

25. The apparatus according to claim 15, characterized in that, The computing module includes: The determination submodule is configured to determine, for any computing unit, the target position of the second target data to be processed in the first target data; The second acquisition submodule is configured to acquire the second target data that the computing unit needs to process from the memory space storing the first target data according to the target location; The second calculation submodule is configured to perform calculations on the second target data through any of the calculation units.

26. The apparatus according to claim 25, characterized in that, The target location includes: each dimension in which the second target data is located; and, for any dimension, the offset and data length of the second target data in that dimension.

27. The apparatus according to claim 15, characterized in that, The at least two computational units belong to different types of operators.

28. The apparatus according to any one of claims 15 to 27, characterized in that, The computational unit is a pre-compiled operator.

29. An operator computing device, characterized in that, include: At least one memory for storing programs; At least one processor is configured to execute a program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to perform the method as described in any one of claims 1-14.

30. An operator computing device, characterized in that, It includes the apparatus according to any one of claims 15-28.

31. An operator computation system, characterized in that, This includes operator computing devices and operator compilation devices; The operator computing device includes the apparatus according to any one of claims 15-28; The operator compilation device is used to compile publishable computational unit packages; The operator computing device is used to import the computing unit package.

32. A computer storage medium storing instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-14.

33. A computer program product comprising instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-14.

34. A chip, characterized in that, Includes at least one processor and interface; The interface is used to provide program instructions or data to the at least one processor; The at least one processor is configured to execute the program line instructions to implement the method as described in any one of claims 1-14.

Images