Executable file distribution methods, devices and electronic equipment

Abstract

This disclosure provides an executable file distribution method, apparatus, and electronic device. The method includes: determining N operator unit groups based on a neural network model; where N is an integer greater than or equal to 1, and each operator unit group includes M operator units in the neural network model that satisfy preset fusion conditions, where M is an integer greater than or equal to 2; generating a fused operator executable file for each of the N operator unit groups to obtain N fused operator executable files; and distributing the N fused operator executable files to a chip. This disclosure reduces the number of generated executable files, thereby reducing the number of times executable files are distributed, shortening the time consumed in distributing executable files, and improving chip performance.

Description

Technical Field

[0001] This disclosure relates to the field of neural network model technology, and in particular to an executable file distribution method, apparatus, and electronic device. Background Technology

[0002] Currently, for neural network models, compilers typically generate executable files for individual operator units and then distribute these executable files to the chip for execution. When there are many operator units in the neural network model, the number of times the executable file is distributed is high, which takes a long time and reduces chip performance. Summary of the Invention

[0003] To address the aforementioned technical problems, this disclosure is proposed. Embodiments of this disclosure provide an executable file distribution method, apparatus, and electronic device.

[0004] According to one aspect of the embodiments of this disclosure, an executable file distribution method is provided, including:

[0005] Based on the neural network model, N operator unit groups are determined; where N is an integer greater than or equal to 1, and each operator unit group includes: M operator units in the neural network model that satisfy the preset fusion conditions, where M is an integer greater than or equal to 2;

[0006] Generate a fusion operator executable file for each of the N operator unit groups to obtain N fusion operator executable files;

[0007] The executable files of the N fusion operators are sent to the chip.

[0008] According to another aspect of the present disclosure, an executable file distribution apparatus is provided, comprising:

[0009] The determination module is used to determine N operator unit groups based on the neural network model; where N is an integer greater than or equal to 1, and each operator unit group includes M operator units in the neural network model that satisfy a preset fusion condition, where M is an integer greater than or equal to 2;

[0010] The generation module is used to generate a fusion operator executable file for each of the N operator unit groups to obtain N fusion operator executable files;

[0011] The distribution module is used to distribute the N fusion operator executable files to the chip.

[0012] According to another aspect of the present disclosure, an electronic device is provided, comprising:

[0013] Memory, used to store computer program products;

[0014] The processor is configured to execute a computer program product stored in the memory, and when the computer program product is executed, to implement the above-described executable file distribution method.

[0015] According to another aspect of the present disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, wherein when the computer program instructions are executed by a processor, the above-described executable file distribution method is implemented.

[0016] According to another aspect of the present disclosure, a computer program product is provided, including computer program instructions that, when executed by a processor, implement the above-described executable file distribution method.

[0017] In the embodiments of this disclosure, N operator unit groups can be determined based on a neural network model. A fusion operator executable file is generated for each of the N operator unit groups, and the N fusion operator executable files corresponding to the N operator unit groups are sent to the chip. It can be seen that, for the neural network model, the embodiments of this disclosure do not generate executable files according to a single operator unit, but according to an operator unit group (which includes at least two operator units). This helps to reduce the number of executable files generated, thereby reducing the number of times executable files are sent, shortening the time consumed by sending executable files, and improving chip performance.

[0018] The technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0019] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0020] Figure 1 This is a flowchart illustrating an exemplary embodiment of the present disclosure of an executable file distribution method.

[0021] Figure 2 This is a flowchart illustrating an executable file distribution method provided in another exemplary embodiment of this disclosure.

[0022] Figure 3 This is a schematic diagram of the overall structure of a neural network model in an exemplary embodiment of this disclosure.

[0023] Figure 4This is a partial structural diagram of a neural network model in an exemplary embodiment of this disclosure.

[0024] Figure 5 This is a partial structural diagram of a neural network model in another exemplary embodiment of this disclosure.

[0025] Figure 6 This is a flowchart illustrating an executable file distribution method provided in another exemplary embodiment of this disclosure.

[0026] Figure 7 This is a flowchart illustrating an executable file distribution method provided in yet another exemplary embodiment of this disclosure.

[0027] Figure 8 This is a schematic diagram illustrating the generation principle of the fusion operator executable file in an exemplary embodiment of this disclosure.

[0028] Figure 9 This is a schematic diagram illustrating the use of a custom extension instruction file in an exemplary embodiment of this disclosure.

[0029] Figure 10 This is a schematic diagram of the structure of an executable file distribution device provided in an exemplary embodiment of this disclosure.

[0030] Figure 11 This is a structural diagram of an electronic device provided in an exemplary embodiment of this disclosure. Detailed Implementation

[0031] Hereinafter, exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present disclosure, and not all embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0032] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0033] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of this disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0034] It should also be understood that in the embodiments disclosed herein, "multiple" can refer to two or more, and "at least one" can refer to one, two or more.

[0035] It should also be understood that any component, data or structure mentioned in the embodiments of this disclosure can generally be understood as one or more unless expressly defined or given to the contrary in the context.

[0036] Furthermore, the term "and / or" in this disclosure is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this disclosure generally indicates that the preceding and following related objects have an "or" relationship.

[0037] It should also be understood that the description of the various embodiments in this disclosure emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0038] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0039] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0040] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0041] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0042] The embodiments disclosed herein can be applied to electronic devices such as terminal devices, computer systems, and servers, and can operate together with a wide range of other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, and servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems, etc.

[0043] Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0044] Exemplary methods

[0045] Figure 1 This is a flowchart illustrating an exemplary embodiment of the present disclosure of an executable file distribution method. Figure 1 The method shown includes steps 110, 120 and 130, which are explained below.

[0046] Step 110: Based on the neural network model, determine N operator unit groups; where N is an integer greater than or equal to 1, and any operator unit group includes M operator units in the neural network model that satisfy the preset fusion conditions, where M is an integer greater than or equal to 2.

[0047] It should be noted that a neural network model can be considered as a sequence of operator units. That is, a neural network model can include multiple operator units (e.g., 50, 70, 100, etc.) arranged in a certain order. Among them, multiple operator units include, but are not limited to, convolution (Conv) operator units, pooling operator units, deconvolution operator units, rectified linear unit (ReLU) operator units, batch normalization (BN) operator units, etc.

[0048] Optionally, the operator unit can also be called a network layer; the neural network model can be a deep neural network model; the neural network model can be used to realize clothing detection, pedestrian detection, lane line detection, target segmentation, etc.

[0049] In step 110, the neural network model can be analyzed to determine which operator units in the neural network model meet the preset fusion conditions, and the neural network model can be grouped according to this to determine N operator unit groups, where each operator unit group includes M operator units that meet the preset fusion conditions.

[0050] Optionally, N can be 1, 2, 3 or an integer greater than 3; M can be 2, 3, 4 or an integer greater than 4; N and M can have the same or different values.

[0051] Optionally, if N is greater than 1, the number M of operator units included in different operator unit groups can be the same or different.

[0052] Step 120: Generate a fusion operator executable file for each of the N operator unit groups to obtain N fusion operator executable files.

[0053] In step 120, the compiler can determine the file generation order for the N operator units, and then generate a fused operator executable file for the operator unit group that is first in the file generation order, followed by the operator unit group that is second in the file generation order, and so on, until N fused operator executable files are obtained; there is a one-to-one correspondence between the N fused operator executable files and the N operator unit groups.

[0054] Alternatively, the compiler can be an Artificial Intelligence (AI) compiler, such as a Low Level Virtual Machine (LLVM) compiler.

[0055] Optionally, the fusion operator executable file corresponding to any operator unit group can refer to an executable file that can be loaded and executed by a chip (e.g., an AI chip) to implement the operational logic of all operator units in that operator unit group.

[0056] Step 130: Distribute the N fusion operator executable files to the chip.

[0057] In step 130, the compiler can determine the order in which the N fusion operator executable files are distributed, and the driver distributes the N fusion operator executable files to the chip in the order of distribution. In this way, the chip can load and execute the N fusion operator executable files.

[0058] In the embodiments of this disclosure, N operator unit groups can be determined based on a neural network model. A fusion operator executable file is generated for each of the N operator unit groups, and the N fusion operator executable files corresponding to the N operator unit groups are sent to the chip. It can be seen that, for the neural network model, the embodiments of this disclosure do not generate executable files according to a single operator unit, but according to an operator unit group (which includes at least two operator units). This helps to reduce the number of executable files generated, thereby reducing the number of times executable files are sent, shortening the time consumed by sending executable files, and improving chip performance.

[0059] exist Figure 1 Based on the illustrated embodiments, as Figure 2 As shown, step 110 includes steps 1101, 1103, 1105 and 1107.

[0060] Step 1101: Determine the distribution information of each operator unit in the neural network model to obtain the first determination result.

[0061] Optionally, the distribution information of any operator unit in the neural network model may include information that characterizes which operator units are adjacent to the operator unit.

[0062] Step 1103: Determine the Synchronous Dynamic Random-Access Memory (SDRAM) corresponding to each operator unit in the neural network model to obtain the second determination result.

[0063] It should be noted that the chip involved in step 130 may include multiple SDRAMs. In specific implementation, the mapping relationship between each operator unit in the neural network model and the corresponding SDRAM in the multiple SDRAMs can be pre-configured.

[0064] In step 1103, for each operator unit in the neural network model, the SDRAM mapped to the operator unit can be determined based on the pre-configured mapping relationship. The determined SDRAM is the SDRAM corresponding to the operator unit.

[0065] Step 1105: Determine the amount of computation result data corresponding to each operator unit in the neural network model to obtain the third determination result.

[0066] It should be noted that, in addition to multiple operator units, a neural network model may also include unit parameter information for each operator unit. Based on the unit parameter information of any operator unit, the amount of data corresponding to the operation result of that operator unit can be determined.

[0067] In an optional example, the neural network model includes a convolution operator unit, and the model also includes unit parameter information for this convolution operator unit. This unit parameter information may specifically include the following parameters: kernel size, kernel stride, padding, specified input width, and specified input height. Referring to the above parameters in this unit parameter information, the data size of the output feature map of the convolution operator unit can be calculated (specifically, a data size value). This data size can then be used as the data size of the operation result corresponding to the convolution operator unit.

[0068] Step 1107: Based on the first determination result, the second determination result, the third determination result and the preset fusion conditions, perform operator unit grouping processing on the neural network model to determine N operator unit groups.

[0069] It should be noted that, referring to the first determination result, we can determine which operator units in the neural network model are adjacent and which are not adjacent; referring to the second determination result, we can determine which operator units in the neural network model correspond to the same SDRAM and which correspond to different SDRAMs; referring to the third determination result, we can determine the amount of data corresponding to the operation result of each operator unit in the neural network model. Thus, combining the first determination result, the second determination result, the third determination result, and the preset fusion conditions, the N operator unit groups obtained after grouping the operator units of the neural network model can satisfy the following:

[0070] M operator units located in the same operator unit group are M consecutive operator units in the neural network model;

[0071] M operator units located in the same operator unit group correspond to the same synchronous dynamic random access memory;

[0072] The sum of the M operation result data corresponding to the M operator units located in the same operator unit group satisfies a preset relationship with the target storage capacity, where the target storage capacity is the storage capacity of the synchronous dynamic random access memory corresponding to any one of the M operator units.

[0073] Optionally, the fact that the sum of the M operation results data and the target storage capacity satisfy a preset relationship can mean that the sum is less than or equal to the target storage capacity; or that the ratio of the sum to the target storage capacity is less than or equal to a preset ratio, which can be a positive number less than or equal to 1, such as 0.7, 0.8, 0.9, etc.

[0074] In an optional example, such as Figure 3 As shown, the neural network model includes 50 operator units arranged in sequence. Assuming that the data volume of the 50 operation results corresponding to these 50 operator units is D1, D2, ..., D50 respectively, and that the SDRAM corresponding to the first 20 operator units is SDRAM1, the SDRAM corresponding to the middle 10 operator units is SDRAM2, and the SDRAM corresponding to the last 20 operator units is SDRAM3, the sum of D1 to D20 is less than the storage capacity of SDRAM1, the sum of D21 to D30 is less than the storage capacity of SDRAM2, and the sum of D31 to D50 is less than the storage capacity of SDRAM3, then it can be determined that the first 20 operator units meet the preset fusion condition, the middle 10 operator units meet the preset fusion condition, and the last 20 operator units also meet the preset fusion condition. At this point, by grouping the neural network model into operator units, three operator unit groups can be determined. The first operator unit group can include the first 20 operator units, the second operator unit group can include the middle 10 operator units, and the third operator unit group can include the last 20 operator units.

[0075] It should be noted that since the sum of D1 to D20 is less than the storage capacity of SDRAM1, SDRAM1 has sufficient space to accommodate the operation results of the first 20 operator units. This helps to avoid the situation where the operation results of some operator units in the first 20 operator units are overwritten. Since the SDRAM corresponding to the first 20 operator units is all SDRAM1, for non-first operator units in the first 20 operator units, the operation results of the corresponding operator unit (e.g., the previous operator unit) can be directly obtained from SDRAM1 for operation, without having to move the operation results of the corresponding operator unit from Double Data Rate Synchronous Dynamic Random-Access Memory (DDR) for operation. This helps to increase throughput and improve chip performance.

[0076] It should be noted that, in specific implementation, other factors can also be considered when grouping operator units. For example, for any operator unit in a neural network model to be the first operator unit, at least one of the following conditions must be met:

[0077] If the second operator unit in the neural network model uses the operation result of the first operator unit, and the number of second operator units is at least two, then the first operator unit and at least two second operator units are located in the same operator unit group;

[0078] If the first operator unit uses the result of the operation of the third operator unit in the neural network model, and the number of the third operator units is at least two, then the first operator unit and at least two third operator units are located in the same operator unit group.

[0079] In an optional example, the local structure reference of the neural network model Figure 4 In this system, Conv1, ReLU1, ReLU2, and Conv2 are each considered as an operator unit. Clearly, ReLU1 and ReLU2 both use the results of Conv1, while Conv2 uses the results of both ReLU1 and ReLU2. Therefore, if Conv1 is the first operator unit, ReLU1 and ReLU2 can each be considered as a second operator unit; if Conv2 is the first operator unit, ReLU1 and ReLU2 can each be considered as a third operator unit. Thus, when performing operator unit grouping, Conv1, ReLU1, ReLU2, and Conv2 can reside in the same operator unit group. Since Conv1, ReLU1, ReLU2, and Conv2 are in the same operator unit group, the sum of the four output data values ​​for each of them can be less than the corresponding SDRAM storage capacity. This helps avoid situations where the results of some operator units within Conv1, ReLU1, ReLU2, and Conv2 are overwritten.

[0080] In another alternative example, the local structure reference of the neural network model Figure 5Let Conv1, ReLU1, ReLU2, ReLU3, Conv2, and Conv3 each be considered as a single operator unit. Clearly, ReLU1, ReLU2, and ReLU3 all use the result of Conv1, while Conv3 uses the results of ReLU3, Conv2, and ReLU2. Therefore, if Conv1 is the first operator unit, ReLU1, ReLU2, and ReLU3 can each be considered as a second operator unit; if Conv3 is the first operator unit, ReLU3, Conv2, and ReLU2 can each be considered as a third operator unit. Thus, when performing operator unit grouping, Conv1, ReLU1, ReLU2, ReLU3, Conv2, and Conv3 can reside in the same operator unit group. Since Conv1, ReLU1, ReLU2, ReLU3, Conv2, and Conv3 are located in the same operator unit group, the sum of the six output data corresponding to Conv1, ReLU1, ReLU2, ReLU3, Conv2, and Conv3 can be less than the storage capacity of the corresponding SDRAM. This helps to avoid the situation where the operation results of some operator units in Conv1, ReLU1, ReLU2, ReLU3, Conv2, and Conv3 are overwritten.

[0081] In the embodiments of this disclosure, by referring to the first determination result, the second determination result, the third determination result and the preset fusion conditions, the neural network model can be reasonably grouped into operator units, which helps to avoid the situation where the operation results of some operator units are overwritten and helps to improve chip performance.

[0082] exist Figure 1 Based on the illustrated embodiments, as Figure 6 As shown, step 110 includes steps 1109, 1111, and 1113.

[0083] Step 1109: Receive the operator unit grouping instruction for the neural network model.

[0084] Optionally, developers can initiate operator unit grouping instructions for the neural network model via voice, touch, or other means. For example, a developer can say "perform operator unit grouping processing on the neural network model" via voice, so that the operator unit grouping instructions for the neural network model can be received in step 1109.

[0085] Step 1111: Obtain grouping method information from the operator unit grouping instruction.

[0086] It should be noted that developers can determine the appropriate grouping information for the neural network model by studying the structure of the neural network model and the hardware characteristics of the chip, combined with experiments. The grouping information can be used to characterize which operator units in the neural network model can be placed in the same operator unit group. For example, the grouping information can be Conv+Reul to characterize that convolution operator units and ReLU operator units can be placed in the same operator unit group. Another example is that the grouping information can be Conv+Reul+BN to characterize that convolution operator units, ReLU operator units, and BN operator units can be placed in the same operator unit group.

[0087] Optionally, the operator unit grouping instruction may include grouping method information, so that in step 1111, the grouping method information can be directly extracted from the operator unit grouping instruction.

[0088] Step 1113: According to the grouping method information, the neural network model is processed by grouping the operator units to determine N operator unit groups.

[0089] In step 1113, the grouping method information can be used as guiding information for the operator unit grouping processing of the neural network model. For example, if the grouping method information is Conv+Reul, and the neural network model only includes a few convolution operator units and a few Reul operator units, then only one operator unit group can be determined. This operator unit group includes all the operator units in the neural network model.

[0090] As can be seen, in the embodiments of this disclosure, the developer can initiate operator unit grouping instructions so that the neural network model can be reasonably grouped into operator units according to the developer's instructions.

[0091] exist Figure 1 Based on the illustrated embodiments, as Figure 7 As shown, step 120 includes steps 1201, 1203, 1205, 1207, and 1209.

[0092] Step 1201: Generate a single operator executable file for each of the M operator units in the target operator unit group to obtain M single operator executable files; wherein, the target operator unit group is any operator unit group among the N operator unit groups.

[0093] In step 1201, M single-operator executable files can be generated by generating executable files in the manner of generating individual operator units; wherein, there can be a one-to-one correspondence between the M single-operator executable files and the M operator units.

[0094] Step 1203: From the neural network model, obtain the unit parameter information for each of the M operator units in the target operator unit group to obtain M unit parameter information.

[0095] Optionally, there can be a one-to-one correspondence between the M unit parameter information and the M operator units. The parameters included in the unit parameter information can be referred to the explanation of step 1105 above, and will not be repeated here.

[0096] Step 1205: Send the M unit parameter information and the M single operator executable files to the chip so that the M unit parameter information is written into the first set of registers of the chip.

[0097] Optionally, the first set of registers may include several general-purpose registers and several special-purpose registers in the chip.

[0098] Step 1207: Obtain the unit parameter information in the first set of registers captured by the simulator, as well as the custom extended instruction files in the M single operator executable files captured by the simulator.

[0099] Optionally, the simulator can be the Spike RISC-V simulator; where Spike is an instruction set simulator and RISC-V is an open-source instruction set architecture based on the principle of reduced instruction set architecture.

[0100] Optionally, the custom extension instruction file can be a RISC-V custom extension instruction file; the custom extension instruction file can include custom extension instruction logic and sequence.

[0101] It should be noted that, as Figure 8 As shown, the operation of a neural network model can be simulated on the chip side using a simulator. During this process, the values ​​in the first set of registers (i.e., unit parameter information) can be captured by the simulator. In addition, custom extended instruction files in each single operator executable file can also be captured by the simulator.

[0102] Step 1209: Based on the unit parameter information captured by the simulator and the custom extended instruction file, generate a fusion operator executable file for the target operator unit group.

[0103] In one specific embodiment, step 1209 includes:

[0104] The unit parameter information captured by the simulator and the custom extended instruction file are concatenated to obtain the concatenation result;

[0105] The concatenated result is then compiled to obtain the final compiled result.

[0106] Based on the compilation results, the fusion operator executable file of the target operator unit group is determined.

[0107] In this implementation, a splicing tool can be used to splice the unit parameter information captured by the simulator and the custom extended instruction file to obtain the spliced ​​result. Next, a compiler can be used to compile the spliced ​​result to obtain the compiled result, which can be binary data. Then, the compiled result can be directly used as the fusion operator executable file for the target operator unit group, or the compiled result can be verified, and if the verification passes, it can be used as the fusion operator executable file for the target operator unit group. Therefore, combining splicing and compilation processes can efficiently and reliably generate the fusion operator executable file.

[0108] In another specific embodiment, step 1209 includes:

[0109] Write the unit parameter information captured by the simulator into the second set of registers;

[0110] Write the cell parameter information from the second set of registers into the third set of registers;

[0111] Based on the unit parameter information in the third set of registers and the custom extended instruction file captured by the simulator, a fusion operator executable file is generated for the target operator unit group.

[0112] Optionally, the second set of registers may include several temporary general-purpose registers that are not frequently used in system resources; the third set of registers may include several special-purpose registers.

[0113] In this implementation, the unit parameter information captured by the simulator can be directly assigned to the dedicated register through a temporary general-purpose register that is not frequently used in system resources, and the custom extended instruction file captured by the simulator can be retained. Then, by combining splicing and compilation processes, the fusion operator executable file can be generated efficiently and reliably.

[0114] It should be noted that the LLVM compiler and Spike RISC-V simulator integrate the RISC-V instruction set and the RISC-V custom extension instruction set. The RISC-V instruction set can include the RISC-V standard instruction set and the RISC-V vector instruction set. The RISC-V custom extension instruction set can include user-defined RISC-V extension instructions, such as the data loading instruction load_cmd, the data storage instruction store_cmd, the convolution calculation instruction tensor_cmd, the pooling calculation instruction pool_cmd, and the linear calculation instruction nl_cmd, etc.

[0115] Compared to the standard RISC-V instruction set, the custom RISC-V instruction set is suitable for large-scale data-parallel computation and data movement. The standard RISC-V instruction set tends to configure different bit fields of general-purpose or special-purpose registers that operate on RISC-V extension instructions. This is because the bit field functionality of general-purpose and special-purpose registers for the RISC-V extension instruction set is quite complex, with a large number of bit fields in a single register, and a single RISC-V extension instruction requires multiple registers to work together, making configuration complex. In the Domain Specific Architecture (DSA) of AI chips, the instructions that actually drive data movement and computation are custom RISC-V extension instructions, not standard RISC-V instructions, as detailed in [link to documentation]. Figure 9 .

[0116] For a fixed network (i.e., a neural network model), network segment (i.e., a local structure within the neural network model), and operator combination (equivalent to the operator unit group mentioned above), the bit fields of the general-purpose registers and special-purpose registers remain unchanged during repeated execution, and the values ​​in the registers also do not change. Based on these characteristics, for the same network, network segment, and operator combination, configuring the general-purpose registers and special-purpose registers once allows for multiple repeated executions. Before and after operator fusion (i.e., before and after generating the fused operator executable file), the bit fields in the general-purpose registers and special-purpose registers do not change, ensuring that the values ​​in the registers also remain unchanged. Furthermore, RISC-V custom extension instructions are only related to the network, network segment, and operator combination; they also remain unchanged across multiple executions.

[0117] In view of this, in the embodiments of this disclosure, the Spike RISC-V simulator can capture and record the following information when running a complete network, network segment, or operator combination for the first time: (1) the values ​​in general-purpose registers and special-purpose registers (equivalent to the unit parameter information captured by the simulator above); (2) RISC-V custom extension instructions (equivalent to the custom extension instruction file captured by the simulator above). Afterwards, automatic generation of fusion operators (i.e., automatic generation of fusion operator executable files) can be performed. During this generation process, the captured values ​​of general-purpose registers can be directly assigned to special-purpose registers through temporary general-purpose registers that are not frequently used in system resources, preserving the original RISC-V custom extension instruction logic and order.

[0118] In the embodiments of this disclosure, the development of fusion operators no longer requires instruction stacking for network models or operator combinations. It only requires including the register values ​​recorded by the Spike RISC-V simulator and a custom RISC-V extension instruction file in the operator file of the fusion operator, such as... Figure 8As shown, the RISC-V backend fusion operator executable file can be generated by compiling the code using a compiler such as LLVM that integrates RISC-V standard instructions and RISC-V custom extension instructions. Thus, the embodiments of this disclosure can quickly and automatically generate fusion operator executable files based on the Spike RISC-V simulator, thereby reducing time and labor costs.

[0119] Any executable file distribution method provided in the embodiments of this disclosure can be executed by any suitable device with data processing capabilities, including but not limited to: terminal devices and servers. Alternatively, any executable file distribution method provided in the embodiments of this disclosure can be executed by a processor, such as by a processor executing any executable file distribution method mentioned in the embodiments of this disclosure by calling corresponding instructions stored in memory. Further details will not be elaborated below.

[0120] Exemplary device

[0121] Figure 10 This is a schematic diagram of the structure of an executable file distribution device provided in an exemplary embodiment of this disclosure. Figure 10 The apparatus shown includes a determining module 1010, a generating module 1020, and a distributing module 1030.

[0122] The determination module 1010 is used to determine N operator unit groups based on the neural network model; where N is an integer greater than or equal to 1, and each operator unit group includes M operator units in the neural network model that satisfy the preset fusion conditions, where M is an integer greater than or equal to 2;

[0123] The generation module 1020 is used to generate a fusion operator executable file for each of the N operator unit groups to obtain N fusion operator executable files;

[0124] The distribution module 1030 is used to distribute N fusion operator executable files to the chip.

[0125] In one optional example, module 1020 is generated, including:

[0126] The first generation submodule is used to generate a single operator executable file for each of the M operator units in the target operator unit group, so as to obtain M single operator executable files; wherein, the target operator unit group is any operator unit group in the N operator unit groups;

[0127] The first acquisition submodule is used to acquire unit parameter information from each of the M operator units in the target operator unit group in the neural network model, so as to obtain M unit parameter information.

[0128] The distribution submodule is used to distribute M unit parameter information and M single operator executable files to the chip, so that the M unit parameter information is written into the first set of registers of the chip;

[0129] The second acquisition submodule is used to acquire the unit parameter information in the first set of registers captured by the simulator, as well as the custom extended instruction files in the M single operator executable files captured by the simulator.

[0130] The second generation submodule is used to generate fusion operator executable files for the target operator unit group based on the unit parameter information captured by the simulator and the custom extended instruction file.

[0131] In one optional example, the second generation submodule includes:

[0132] The splicing unit is used to splice the unit parameter information captured by the simulator and the custom extended instruction file to obtain the splicing result;

[0133] The compilation processing unit is used to compile the concatenated results to obtain the compiled result.

[0134] The determination unit is used to determine the fusion operator executable file of the target operator unit group based on the compilation processing results.

[0135] In one optional example, the second generation submodule includes:

[0136] The first write unit is used to write the unit parameter information captured by the simulator into the second set of registers;

[0137] The second write unit is used to write the unit parameter information in the second set of registers into the third set of registers;

[0138] The generation unit is used to generate a fusion operator executable file for the target operator unit group based on the unit parameter information in the third set of registers and the custom extended instruction file captured by the simulator.

[0139] In one optional example, module 1010 is defined as including:

[0140] The first determining submodule is used to determine the distribution information of each operator unit in the neural network model in order to obtain the first determining result;

[0141] The second determination submodule is used to determine the synchronous dynamic random access memory corresponding to each operator unit in the neural network model, so as to obtain the second determination result;

[0142] The third determination submodule is used to determine the amount of operation result data corresponding to each operator unit in the neural network model, so as to obtain the third determination result;

[0143] The first grouping processing submodule is used to perform operator unit grouping processing on the neural network model based on the first determination result, the second determination result, the third determination result and the preset fusion conditions, so as to determine N operator unit groups.

[0144] In one optional example,

[0145] M operator units located in the same operator unit group are M consecutive operator units in the neural network model;

[0146] M operator units located in the same operator unit group correspond to the same synchronous dynamic random access memory;

[0147] The sum of the M operation result data corresponding to the M operator units located in the same operator unit group satisfies a preset relationship with the target storage capacity, where the target storage capacity is the storage capacity of the synchronous dynamic random access memory corresponding to any one of the M operator units.

[0148] In one optional example, any operator unit in the neural network model is the first operator unit, and at least one of the following is satisfied:

[0149] If the second operator unit in the neural network model uses the operation result of the first operator unit, and the number of second operator units is at least two, then the first operator unit and at least two second operator units are located in the same operator unit group;

[0150] If the first operator unit uses the result of the operation of the third operator unit in the neural network model, and the number of the third operator units is at least two, then the first operator unit and at least two third operator units are located in the same operator unit group.

[0151] In one optional example, module 1010 is defined as including:

[0152] The receiving submodule is used to receive operator unit grouping instructions for neural network models;

[0153] The third acquisition submodule is used to obtain grouping method information from the grouping instructions of the operator unit;

[0154] The second grouping processing submodule is used to group the neural network model into operator units according to the grouping method information to determine N operator unit groups.

[0155] Exemplary electronic devices

[0156] Below, for reference Figure 11This describes an electronic device according to embodiments of the present disclosure. The electronic device may be either or both of a first device and a second device, or a standalone device independent of them, which may communicate with the first device and the second device to receive acquired input signals from them.

[0157] Figure 11 A block diagram of an electronic device 1100 according to an embodiment of the present disclosure is shown.

[0158] like Figure 11 As shown, the electronic device 1100 includes one or more processors 1101 and memory 1102.

[0159] The processor 1101 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 1100 to perform desired functions.

[0160] The memory 1102 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1101 may execute the program instructions to implement the executable file delivery methods of the various embodiments of this disclosure described above and / or other desired functions. Various contents such as input signals, signal components, and noise components may also be stored in the computer-readable storage medium.

[0161] In one example, the electronic device 1100 may also include an input device 1103 and an output device 1104, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0162] For example, when electronic device 1100 is a first device or a second device, the input device 1103 may be a microphone or a microphone array. When electronic device 1100 is a standalone device, the input device 1103 may be a communication network connector for receiving acquired input signals from the first device and the second device.

[0163] In addition, the input device 1103 may also include, for example, a keyboard, a mouse, etc.

[0164] The output device 1104 can output various information to the outside. The output device 1104 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0165] Of course, for the sake of simplicity, Figure 11 Only some of the components of the electronic device 1100 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 1100 may include any other suitable components depending on the specific application.

[0166] Exemplary computer program products and computer-readable storage media

[0167] In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps of the executable file delivery methods according to various embodiments of this disclosure as described in the "Exemplary Methods" section of this specification.

[0168] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0169] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform the steps of the executable file distribution method according to various embodiments of this disclosure as described in the "Exemplary Methods" section above.

[0170] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0171] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0172] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0173] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0174] The methods and apparatus of this disclosure may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of this disclosure are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the methods according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the methods according to this disclosure.

[0175] It should also be noted that in the apparatus, devices, and methods of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.

[0176] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0177] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. An executable file delivery method, characterized in that, The method applied to a simulator comprises: determining N operator unit groups based on a neural network model, wherein N is an integer greater than or equal to 1, any operator unit group comprises M operator units in the neural network model satisfying a preset fusion condition, M is an integer greater than or equal to 2, the M operator units located in the same operator unit group are continuous M operator units in the neural network model, the M operator units located in the same operator unit group correspond to the same synchronous dynamic random memory, and the sum of M operation result data amounts corresponding to the M operator units located in the same operator unit group satisfies a preset relationship with a target storage capacity, the target storage capacity is a storage capacity of the synchronous dynamic random memory corresponding to any operator unit in the M operator units; generating a fusion operator executable file for each of the N operator unit groups to obtain N fusion operator executable files; downloading the N fusion operator executable files to a chip.

2. The method of claim 1, wherein, The method for generating a fusion operator executable file for each of the N operator unit groups comprises: generating a single-operator executable file for each of the M operator units in a target operator unit group to obtain M single-operator executable files, wherein the target operator unit group is any operator unit group in the N operator unit groups; obtaining unit parameter information for each of the M operator units in the target operator unit group from the neural network model to obtain M unit parameter information; downloading the M unit parameter information and the M single-operator executable files to a chip of the simulator, so that the M unit parameter information is written into a first group of registers of the chip of the simulator; obtaining unit parameter information in the first group of registers captured by the simulator and a custom extension instruction file in the M single-operator executable files captured by the simulator; generating a fusion operator executable file for the target operator unit group based on the unit parameter information and the custom extension instruction file captured by the simulator.

3. The method of claim 2, wherein, The method for generating a fusion operator executable file for the target operator unit group based on the unit parameter information and the custom extension instruction file captured by the simulator comprises: splicing the unit parameter information and the custom extension instruction file captured by the simulator to obtain a splicing result; compiling the splicing result to obtain a compilation result; determining a fusion operator executable file for the target operator unit group based on the compilation result.

4. The method of claim 2, wherein, The method for generating a fusion operator executable file for the target operator unit group based on the unit parameter information and the custom extension instruction file captured by the simulator comprises: writing the unit parameter information captured by the simulator into a second group of registers; writing the unit parameter information in the second group of registers into a third group of registers; generating a fusion operator executable file for the target operator unit group based on the unit parameter information in the third group of registers and the custom extension instruction file captured by the simulator.

5. The method according to any one of claims 1 to 4, characterized in that, The neural network model is used to determine N operator unit groups, including: The distribution information of each operator unit in the neural network model is determined respectively to obtain a first determination result; The corresponding synchronous dynamic random memory of each operator unit in the neural network model is determined respectively to obtain a second determination result; The corresponding operation result data volume of each operator unit in the neural network model is determined respectively to obtain a third determination result; The first determination result, the second determination result, the third determination result and the preset fusion condition are used for operator unit grouping processing of the neural network model to determine N operator unit groups.

6. The method of claim 5, wherein, Any operator unit in the neural network model is a first operator unit, and at least one of the following conditions is met: If the second operator unit in the neural network model uses the operation result of the first operator unit, and the number of the second operator unit is at least two, then the first operator unit and at least two second operator units are located in the same operator unit group; If the first operator unit uses the operation result of the third operator unit in the neural network model, and the number of the third operator unit is at least two, then the first operator unit and at least two third operator units are located in the same operator unit group.

7. The method according to any one of claims 1 to 4, characterized in that, The neural network model is used to determine N operator unit groups, including: An operator unit grouping instruction for the neural network model is received; Grouping mode information is obtained from the operator unit grouping instruction; According to the grouping mode information, the neural network model is subjected to operator unit grouping processing to determine N operator unit groups.

8. An executable file delivery apparatus characterized by comprising: The device includes: A determination module is configured to determine N operator unit groups based on a neural network model, wherein N is an integer greater than or equal to 1, any operator unit group includes M operator units in the neural network model that meet a preset fusion condition, and M is an integer greater than or equal to 2; the M operator units located in the same operator unit group are consecutive M operator units in the neural network model; the M operator units located in the same operator unit group correspond to the same synchronous dynamic random memory; the sum of M operation result data volumes corresponding to the M operator units located in the same operator unit group satisfies a preset relationship with a target storage capacity, and the target storage capacity is the storage capacity of the synchronous dynamic random memory corresponding to any operator unit in the M operator units; A generation module is configured to generate a fusion operator executable file for each operator unit group in the N operator unit groups to obtain N fusion operator executable files; A delivery module is configured to deliver the N fusion operator executable files to a chip.

9. An electronic device, comprising: It includes: A memory is configured to store a computer program product; A processor is configured to execute the computer program product stored in the memory, and when the computer program product is executed, the executable file delivery method in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions, when executed by a processor, implement the executable file distribution method in any one of claims 1 to 7.

11. A computer program product comprising computer program instructions, characterised in that, The computer program instructions, when executed by a processor, implement the executable file distribution method in any one of claims 1 to 7.

Images