Heterogeneous multiprocessor neural network operator scheduling method, device, equipment and medium

Abstract

The invention discloses a heterogeneous multiprocessor neural network operator scheduling method and device, equipment and a medium, and relates to the technical field of computers. Comprising the steps that a target neural network model is acquired and analyzed to obtain a data flow diagram, and the data flow diagram is stored in a preset storage space; obtaining input data for the target neural network model, and preprocessing the input data to obtain a corresponding data processing result; creating a data mapping table based on the data processing result and the data flow diagram, and sending the data mapping table to the accelerator; and the accelerator calculates the input data by scheduling an accelerator operator based on the data flow diagram and the data mapping table so as to obtain output data. Therefore, the interaction frequency between the central processing unit and the accelerator is reduced in the process of executing the calculation task, and the resource utilization rate and the calculation efficiency of the whole heterogeneous multi-core processor system are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a heterogeneous multi-processor neural network operator scheduling method, device, equipment and medium. Background Art

[0002] With the widespread application of deep learning neural networks in computer vision, natural language processing, and speech recognition, the scale and complexity of neural networks continue to increase. This growth places extremely high demands on computing resources, especially in tasks involving large-scale data and complex models. Traditional single-processor architectures often have difficulty meeting the required computing power and efficiency. To solve this problem, heterogeneous multi-core processor architectures have gradually become a research hotspot.

[0003] In the heterogeneous multi-core processor system for deep learning tasks, the central processing unit usually plays a core role in task scheduling, data transmission and management, while accelerators such as digital signal processors are responsible for efficiently executing computationally intensive tasks. Although heterogeneous architecture can significantly improve parallel processing capabilities, frequent communication and scheduling operations between the central processing unit and accelerators often lead to large performance overhead, especially in large-scale data processing tasks. Frequent scheduling and data transmission will slow down the execution efficiency of the entire system. In addition, traditional scheduling methods usually rely on dynamic data distribution and scheduling, which will introduce unnecessary communication delays and resource waste during multiple transmissions, causing the central processing unit to frequently turn on the computing core, increasing the overall core ignition time of the system.

[0004] As can be seen from the above, how to reduce the interaction frequency between the central processing unit and the accelerator in the process of executing computing tasks and improve the resource utilization and computing efficiency of the entire heterogeneous multi-core processor system is an urgent problem to be solved. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a heterogeneous multi-processor neural network operator scheduling method, device, equipment and medium, which can reduce the interaction frequency between the central processing unit and the accelerator in the process of executing computing tasks, and improve the resource utilization and computing efficiency of the entire heterogeneous multi-core processor system. The specific scheme is as follows:

[0006] In a first aspect, the present application provides a heterogeneous multi-processor neural network operator scheduling method, which is applied to a central processing unit as a host side and an accelerator as a device side, including:

[0007] Acquire and parse the target neural network model to obtain a data flow graph, and store the data flow graph in a preset storage space;

[0008] Acquire input data for the target neural network model, and preprocess the input data to obtain corresponding data processing results;

[0009] Creating a data mapping table based on the data processing result and the data flow graph, and sending the data mapping table to the accelerator;

[0010] The accelerator calculates the input data by scheduling an accelerator operator based on the data flow graph and the data mapping table to obtain output data.

[0011] Optionally, acquiring and parsing the target neural network model to obtain a data flow graph, and storing the data flow graph in a preset storage space, includes:

[0012] Parsing the target neural network model and obtaining the execution order of each computing node in the data flow graph of the target neural network model;

[0013] Parsing the target neural network model and obtaining the operator type of each computing node in the data flow graph of the target neural network model;

[0014] Parsing the target neural network model and obtaining input and output data of each neural network operator in the data flow graph of the target neural network model;

[0015] Storing the execution order of each computing node in the data flow graph into a first preset storage space;

[0016] Storing the operator type of each computing node in the data flow graph into a second preset storage space;

[0017] The input and output data of each neural network operator in the data flow graph are stored in a third preset storage space.

[0018] Optionally, the acquiring input data for the target neural network model and preprocessing the input data to obtain corresponding data processing results includes:

[0019] Acquiring input data for the target neural network model, and performing data analysis on the input data to obtain a data analysis result;

[0020] The input data is divided into blocks according to the data analysis result, and a corresponding target block index table is generated according to the block result of the input data;

[0021] A corresponding computing resource allocation table is created according to the target block index table, and the input data, the target block index table and the computing resource allocation table are merged to obtain a data processing result.

[0022] Optionally, the step of dividing the input data into blocks according to the data analysis result, and generating a corresponding target block index table according to the block result of the input data includes:

[0023] Divide the input data into blocks according to the data analysis result, and determine whether there is a non-zero element in each block;

[0024] If the non-zero element does not exist in the block, the block is marked as a first target block; if the non-zero element exists in the block, the block is marked as a second target block;

[0025] The marking result is updated to the initial block index table to obtain a target block index table corresponding to the block division result of the input data.

[0026] Optionally, creating a corresponding computing resource allocation table according to the target block index table includes:

[0027] Determine whether the number of the first target blocks in the target block index table is greater than a preset number threshold, and distribute the input data based on the determination result to obtain a corresponding distribution result;

[0028] Creating a corresponding computing resource allocation table according to the allocation result;

[0029] Wherein, allocating the input data based on the judgment result includes:

[0030] If the number of the first target blocks is greater than the preset number threshold, allocating the input data to a first preset calculation unit;

[0031] If the number of the first target blocks is not greater than the preset number threshold, allocating the input data to a second preset calculation unit;

[0032] The computing capability of the first preset computing unit is smaller than that of the second preset computing unit.

[0033] Optionally, the creating a data mapping table based on the data processing result and the data flow graph includes:

[0034] Based on the data processing result, the input and output data of the neural network operators corresponding to the respective computing nodes are determined from the third preset storage space, and a data mapping table is created.

[0035] Optionally, the accelerator calculates the input data by scheduling an accelerator operator based on the data flow graph and the data mapping table to obtain output data, including:

[0036] The accelerator obtains the computing node index based on the data flow graph according to the execution order of each computing node stored in the first preset storage space;

[0037] The accelerator obtains, from the second preset storage space based on the computing node index, a target operator type corresponding to the computing node in the data flow graph;

[0038] The accelerator obtains input and output data of the target neural network operator corresponding to the computing node in the data flow graph from the third preset storage space based on the computing node index and the data mapping table;

[0039] The accelerator calls a corresponding accelerator operator based on the target operator type and the input and output data of the target neural network operator;

[0040] The accelerator operator determines a target computing unit for computing current input data according to the computing resource allocation table, so that the target computing unit computes the current input data to obtain output data.

[0041] Optionally, the target computing unit computes the current input data to obtain output data, including:

[0042] The target calculation unit calculates the second target block in the current input data based on the target block index table to obtain output data.

[0043] In a second aspect, the present application provides a heterogeneous multi-processor neural network operator scheduling device, which is applied to a central processing unit as a host side and an accelerator as a device side, including:

[0044] A neural network model parsing module, used to acquire and parse the target neural network model to obtain a data flow graph, and store the data flow graph in a preset storage space;

[0045] An input data preprocessing module, used to obtain input data for the target neural network model and preprocess the input data to obtain corresponding data processing results;

[0046] A data mapping table creation module, used to create a data mapping table based on the data processing result and the data flow graph, and send the data mapping table to the accelerator;

[0047] An output data calculation module is used for the accelerator to calculate the input data by scheduling an accelerator operator based on the data flow graph and the data mapping table to obtain output data.

[0048] In a third aspect, the present application provides an electronic device, including:

[0049] Memory, used to store computer programs;

[0050] A processor is used to execute the computer program to implement the aforementioned heterogeneous multi-processor neural network operator scheduling method.

[0051] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned heterogeneous multi-processor neural network operator scheduling method.

[0052] The present application provides a heterogeneous multi-processor neural network operator scheduling method, which obtains and parses a target neural network model to obtain a data flow graph, and stores the data flow graph in a preset storage space; obtains input data for the target neural network model, and preprocesses the input data to obtain corresponding data processing results; creates a data mapping table based on the data processing results and the data flow graph, and sends the data mapping table to the accelerator; the accelerator calculates the input data based on the data flow graph and the data mapping table by scheduling an accelerator operator to obtain output data.

[0053] As can be seen from the above, the present application analyzes the target neural network model to obtain a data flow graph, and sends the data flow graph to the accelerator through the data mapping table, so that the accelerator calculates the input data based on the data flow graph and the data mapping table by scheduling the accelerator operator, thereby avoiding the central processing unit from frequently scheduling the data in the target neural network model to the accelerator, thereby reducing the frequency of interaction between the central processing unit and the accelerator during the execution of computing tasks, and improving the resource utilization and computing efficiency of the entire heterogeneous multi-core processor system. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0055] Figure 1 A flowchart of a method for scheduling neural network operators for heterogeneous multi-processors used in a central processing unit as a host and an accelerator as a device disclosed in the present application;

[0056] Figure 2 A flowchart of a method for scheduling a neural network operator of a heterogeneous multi-processor specifically applied to a central processing unit as a host side and an accelerator as a device side disclosed in the present application;

[0057] Figure 3 A schematic diagram of a heterogeneous multi-processor neural network operator scheduling device disclosed in the present application, which is applied to a central processing unit as a host side and an accelerator as a device side;

[0058] Figure 4 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] In recent years, with the widespread application of deep learning neural networks in fields such as computer vision, natural language processing, and speech recognition, the scale and complexity of neural networks have continued to increase. This growth places extremely high demands on computing resources, especially in tasks involving large-scale data and complex models. Traditional single-processor architectures are often difficult to meet the required computing power and efficiency. In order to solve this problem, heterogeneous multi-processor architectures have gradually become a research hotspot. Traditional scheduling methods usually rely on dynamic data distribution and scheduling, which will introduce unnecessary communication delays and resource waste during multiple transmissions, resulting in the need for the CPU to frequently turn on the computing core, increasing the overall core ignition time of the system. To this end, the present application provides a heterogeneous multi-processor neural network operator scheduling scheme that can reduce the frequency of interaction between the central processing unit and the accelerator during the execution of computing tasks, and improve the resource utilization and computing efficiency of the entire heterogeneous multi-core processor system.

[0061] See also Figure 1 As shown, the embodiment of the present application discloses a heterogeneous multi-processor neural network operator scheduling method, which is applied to a central processing unit as a host end and an accelerator as a device end, including:

[0062] Step S11, obtain and parse the target neural network model to obtain a data flow graph, and store the data flow graph in a preset storage space.

[0063] In this embodiment, the central processing unit loads the target neural network model through a preset interface, and parses the target network model to obtain a data flow graph corresponding to the target neural network. Wherein, the target neural network model includes but is not limited to a computer vision deep learning neural network model, a natural language processing deep learning neural network model, and a speech recognition deep learning neural network model. Different data in the data flow graph are stored in corresponding storage spaces respectively. Wherein, the different data in the data flow graph include the execution order of each computing node, the operator type, and the input and output data space of the target neural network operator. In addition, the storage space can be specifically determined according to the actual application situation. For example, in a specific implementation, the data in the data flow graph can be saved in a pre-created array. Specifically, the execution order of each computing node in the data flow graph is saved in the first array, the operator type of each computing node in the data flow graph is saved in the second array, the input and output data space of the target neural network operator in the data flow graph is saved in the third array, and the input and output data space corresponding to each computing node is saved in the fourth array corresponding to the index in the third array.

[0064] Furthermore, after the data flow graph is stored in a preset storage space, the data flow graph needs to be mapped to an accelerator so that the accelerator executes the calculation steps through the data flow graph. For example, in a specific implementation, the data flow graph can be mapped to the accelerator by establishing a corresponding structure based on the data in the data flow graph. Specifically, the creation of a data mapping table based on the data processing result and the data flow graph may include: creating a corresponding first structure based on the execution order stored in the first array, and creating a corresponding second structure based on the operator type stored in the second array.

[0065] Step S12: Obtain input data for the target neural network model, and preprocess the input data to obtain corresponding data processing results.

[0066] In this embodiment, the central processing unit obtains input data for the target neural network model through a preset interface, performs targeted sparse optimization to reduce the scale of calculation, and divides the computing tasks into thread levels through simple preprocessing to achieve the purpose of load balancing.

[0067] Furthermore, after preprocessing the input data, the input data needs to be mapped to an accelerator so that the accelerator performs calculation steps on the input data. For example, in a specific implementation, the input data can be mapped to the accelerator by establishing a corresponding structure based on the input data. Specifically, creating a data mapping table based on the data processing result and the data flow graph may include: creating a corresponding third structure based on the input and output data of the neural network operator stored in the third array.

[0068] Step S13: Create a data mapping table based on the data processing result and the data flow graph, and send the data mapping table to the accelerator.

[0069] In this embodiment, the central processing unit maps the data flow graph to the accelerator through the data mapping table so that the accelerator can calculate the input data. For example, in a specific implementation, the data mapping table can be mapped to the accelerator by establishing a corresponding structure. Specifically, the creation of a data mapping table based on the data processing results and the data flow graph can include: creating a fourth structure based on the first structure and the third structure by calling related functions; creating a data mapping table based on the fourth structure; wherein the fourth structure contains the correspondence between each computing node and the input and output data of the neural network operator. That is, by creating the structure and copying the structure to the accelerator, the data transmission efficiency can be improved and the transmission load between the central processing unit and the accelerator can be reduced.

[0070] Step S14: The accelerator calculates the input data by scheduling an accelerator operator based on the data flow graph and the data mapping table to obtain output data.

[0071] In this embodiment, the accelerator obtains the static address of the input and output data of each operator in the data flow graph through the data mapping table, and calls the operator corresponding to the current computing node during the calculation process. Specifically, the process in which the accelerator calculates the input data based on the data mapping table to obtain the output data may include: the accelerator obtains the input and output data of the corresponding neural network operator from the third structure according to the corresponding relationship between the input and output data of the computing nodes stored in the fourth structure and the neural network operator based on the execution order of the computing nodes stored in the first structure based on the fourth structure, and calls the corresponding accelerator computing operator to calculate the input data according to the operator type of the computing nodes stored in the second structure.

[0072] As can be seen from the above, the embodiment of the present application analyzes the target neural network model to obtain a data flow graph, and sends the data flow graph to the accelerator through the data mapping table, so that the accelerator calculates the input data based on the data flow graph and the data mapping table by scheduling the accelerator operator, thereby avoiding the central processing unit from frequently scheduling the data in the target neural network model to the accelerator, thereby reducing the frequency of interaction between the central processing unit and the accelerator during the execution of computing tasks, thereby improving the resource utilization and computing efficiency of the entire heterogeneous multi-core processor system.

[0073] See also Figure 2 As shown, in order to reduce the load and overhead of data calculation, the present application provides a specific heterogeneous multi-processor neural network operator scheduling method, which is applied to a central processing unit, including:

[0074] Step S21, obtain and parse the target neural network model to obtain a data flow graph, and store the data flow graph in a preset storage space.

[0075] Step S22: Obtain input data for the target neural network model, and perform data analysis on the input data to obtain data analysis results.

[0076] In this embodiment, the central processing unit obtains input data for the target neural network model through a preset target interface, and evaluates the calculation scale of the current input data by analyzing the distribution of the input data.

[0077] Step S23: block the input data according to the data analysis result, and generate a corresponding target block index table according to the block result of the input data.

[0078] In this embodiment, the central processing unit performs a block operation on the input data according to a certain multiple of the convolution kernel size. For example, if the size of the convolution kernel is 3*3, and the input data is block-operated according to 100 times the size of the convolution kernel, the size of the resulting data block is 300*300. Further, the data blocks are classified so as to generate a corresponding target block index table based on the obtained classification results. For example, in a specific implementation, the data blocks can be classified by calculating the number of non-zero elements in each data block. Specifically, the input data is divided into blocks according to the data analysis result, and the corresponding target block index table is generated according to the block result of the input data, which may include: dividing the input data into blocks according to the convolution kernel size, and judging whether there is a non-zero element in each block; if the non-zero element does not exist in the block, the corresponding position of the block in the initial block index table is marked as 0, and if the non-zero element exists in the block, the corresponding position of the block in the initial block index table is marked as 1, so as to obtain a target block index table corresponding to the block result of the input data. Further, in the process of performing convolution calculation, by detecting the target block index table, it is judged whether the position corresponding to the current data block is 0, if the corresponding position of the current data block in the target block index table is 0, the convolution calculation of the current data block is skipped; if the corresponding position of the current data block in the target block index table is 1, the convolution calculation is performed on the current data block. By performing block processing on the input data and generating the target block index table, invalid calculations can be skipped, thereby improving communication efficiency and utilization of computing resources.

[0079] Step S24: Create a corresponding computing resource allocation table according to the target block index table, and merge the input data, the target block index table and the computing resource allocation table to obtain a data processing result.

[0080] In this embodiment, the central processing unit allocates computing resources through the target block index table and creates a corresponding computing resource allocation table, so that the accelerator mobilizes the computing unit used to calculate the input data through the computing resource allocation table. Specifically, the creation of the corresponding computing resource allocation table according to the target block index table may include: judging whether the number of the first target blocks in the target block index table is greater than the preset number threshold, and allocating the input data based on the judgment result to obtain the corresponding allocation result; creating the corresponding computing resource allocation table according to the allocation result; wherein, the allocation of the input data based on the judgment result may include: if the number of the first target blocks is greater than the preset number threshold, the input data is allocated to the first preset computing unit; if the number of the first target blocks is not greater than the preset number threshold, the input data is allocated to the second preset computing unit; wherein the computing power of the first preset computing unit is less than that of the second preset computing unit. That is, the input data is tested for sparsity through the computing allocation table, wherein the more blocks with the corresponding position of 0 in the target block index table, the higher the sparsity. The CPU will allocate fewer computing units or threads to sparser data blocks for computing. By establishing a corresponding computing resource allocation table to divide computing tasks into threads, load balancing can be achieved.

[0081] Step S25: Create a data mapping table based on the data processing result and the data flow graph, and send the data mapping table to the accelerator.

[0082] Step S26: The accelerator calculates the input data by scheduling an accelerator operator based on the data flow graph and the data mapping table to obtain output data.

[0083] Among them, the specific implementation process of the above steps S21, S25 and S26 can refer to the corresponding content in the previous embodiment, and will not be repeated here.

[0084] As can be seen from the above, the embodiment of the present application analyzes the target neural network model to obtain a data flow graph, and sends the data flow graph to the accelerator through the data mapping table, so that the accelerator completes the subsequent calculation steps through the data mapping table, which can avoid the central processing unit from frequently scheduling the data in the target neural network model to the accelerator. And by dividing the input data into blocks, and generating a corresponding target block index table according to the block results of the input data, and creating a corresponding computing resource allocation table according to the target block index table. Targeted sparse optimization is performed on the input data to reduce the calculation scale, and the computing tasks are divided into thread levels through simple preprocessing to achieve the purpose of load balancing. Thereby, the interaction frequency between the central processing unit and the accelerator can be reduced in the process of executing computing tasks, and the resource utilization and computing efficiency of the entire heterogeneous multi-core processor system can be improved.

[0085] See also Figure 3 As shown, the embodiment of the present application discloses a heterogeneous multi-processor neural network operator scheduling device, including:

[0086] A neural network model parsing module 11 is used to acquire and parse the target neural network model to obtain a data flow graph, and store the data flow graph in a preset storage space;

[0087] An input data preprocessing module 12 is used to obtain input data for the target neural network model and preprocess the input data to obtain corresponding data processing results;

[0088] A data mapping table creation module 13, used to create a data mapping table based on the data processing result and the data flow graph, and send the data mapping table to the accelerator;

[0089] The output data calculation module 14 is used for the accelerator to calculate the input data based on the data flow graph and the data mapping table by scheduling the accelerator operator to obtain output data.

[0090] As can be seen from the above, the embodiment of the present application analyzes the target neural network model to obtain a data flow graph, and sends the data flow graph to the accelerator through the data mapping table, so that the accelerator calculates the input data based on the data flow graph and the data mapping table by scheduling the accelerator operator, thereby avoiding the central processing unit from frequently scheduling the data in the target neural network model to the accelerator, thereby reducing the frequency of interaction between the central processing unit and the accelerator during the execution of computing tasks, thereby improving the resource utilization and computing efficiency of the entire heterogeneous multi-core processor system.

[0091] In some specific embodiments, the neural network model parsing module 11 may specifically include:

[0092] An execution order acquisition unit, used to parse the target neural network model and acquire the execution order of each computing node in the data flow graph of the target neural network model;

[0093] An operator type acquisition unit, used to parse the target neural network model and acquire the operator type of each computing node in the data flow graph of the target neural network model;

[0094] An input and output data acquisition unit, used to parse the target neural network model and acquire input and output data of each neural network operator in the data flow graph of the target neural network model;

[0095] A first data storage unit, used for storing the execution order of each computing node in the data flow graph into a first preset storage space;

[0096] A second data storage unit, used to store the operator type of each computing node in the data flow graph into a second preset storage space;

[0097] The third data storage unit is used to store the input and output data of each neural network operator in the data flow graph into a third preset storage space.

[0098] In some specific embodiments, the input data preprocessing module 12 may specifically include:

[0099] A data analysis submodule, used to obtain input data for the target neural network model and perform data analysis on the input data to obtain data analysis results;

[0100] A target block index table generation submodule, used to divide the input data into blocks according to the data analysis result, and generate a corresponding target block index table according to the block result of the input data;

[0101] The computing resource allocation table generation submodule is used to create a corresponding computing resource allocation table according to the target block index table, and merge the input data, the target block index table and the computing resource allocation table to obtain a data processing result.

[0102] In some specific embodiments, the data mapping table creation module 13 may specifically include:

[0103] A data mapping table creation unit is used to determine the input and output data of the neural network operator corresponding to each computing node from the third preset storage space based on the data processing result, and create a data mapping table.

[0104] In some specific embodiments, the output data calculation module 14 may specifically include:

[0105] A computing node index obtaining unit, configured for the accelerator to obtain computing node indexes according to the execution order of the computing nodes stored in the first preset storage space based on the data flow graph;

[0106] A target operator type acquisition unit, configured for the accelerator to acquire, from the second preset storage space based on the computing node index, a target operator type corresponding to a computing node in the data flow graph;

[0107] A target input and output data acquisition unit, wherein the accelerator acquires input and output data of a target neural network operator corresponding to a computing node in the data flow graph from the third preset storage space based on the computing node index and the data mapping table;

[0108] An accelerator operator scheduling unit, configured for the accelerator to call a corresponding accelerator operator based on the target operator type and the input and output data of the target neural network operator;

[0109] An input data computing unit is used for the accelerator operator to determine a target computing unit for computing current input data according to the computing resource allocation table, so that the target computing unit computes the current input data to obtain output data.

[0110] In some specific embodiments, the target block index table generation submodule may specifically include:

[0111] A data block division unit, used for dividing the input data into blocks according to the data analysis result, and determining whether there is a non-zero element in each block;

[0112] a target block marking unit, configured to mark the block as a first target block if the non-zero element does not exist in the block, and mark the block as a second target block if the non-zero element exists in the block;

[0113] The target block index table generating unit is used to update the marking result to the initial block index table to obtain a target block index table corresponding to the block division result of the input data.

[0114] In some specific embodiments, the computing resource allocation table generating submodule may specifically include:

[0115] an allocation result determination unit, configured to determine whether the number of the first target blocks in the target block index table is greater than a preset number threshold, and to allocate the input data based on the determination result to obtain a corresponding allocation result; wherein the allocation of the input data based on the determination result comprises: if the number of the first target blocks is greater than the preset number threshold, allocating the input data to a first preset calculation unit; if the number of the first target blocks is not greater than the preset number threshold, allocating the input data to a second preset calculation unit; wherein the computing power of the first preset calculation unit is less than that of the second preset calculation unit;

[0116] The computing resource allocation table creation unit is used to create a corresponding computing resource allocation table according to the allocation result.

[0117] Furthermore, the present application also discloses an electronic device. Figure 4 It is a structural diagram of an electronic device 20 shown according to an exemplary embodiment, and the content in the figure cannot be regarded as any limitation on the scope of use of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input and output interface 25 and a communication bus 26. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the heterogeneous multi-processor neural network operator scheduling method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment can specifically be an electronic computer.

[0118] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0119] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0120] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the heterogeneous multi-processor neural network operator scheduling method executed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0121] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the heterogeneous multi-processor neural network operator scheduling method disclosed above is implemented. The specific steps of the method can refer to the corresponding contents disclosed in the above embodiments, and will not be repeated here.

[0122] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0123] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0124] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0125] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0126] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for scheduling operators of heterogeneous multi-processor neural networks, characterized in that: Applied to host-side CPUs and device-side accelerators, including: Acquire and parse the target neural network model to obtain a data flow graph, and store the data flow graph in a preset storage space; Acquire input data for the target neural network model, and preprocess the input data to obtain corresponding data processing results; Creating a data mapping table based on the data processing result and the data flow graph, and sending the data mapping table to the accelerator; The accelerator calculates the input data by scheduling an accelerator operator based on the data flow graph and the data mapping table to obtain output data.

2. The heterogeneous multi-processor neural network operator scheduling method according to claim 1, characterized in that: The acquiring and parsing the target neural network model to obtain a data flow graph, and storing the data flow graph in a preset storage space, includes: Parsing the target neural network model and obtaining the execution order of each computing node in the data flow graph of the target neural network model; Parsing the target neural network model and obtaining the operator type of each computing node in the data flow graph of the target neural network model; Parsing the target neural network model and obtaining input and output data of each neural network operator in the data flow graph of the target neural network model; Storing the execution order of each computing node in the data flow graph into a first preset storage space; Storing the operator type of each computing node in the data flow graph into a second preset storage space; The input and output data of each neural network operator in the data flow graph are stored in a third preset storage space.

3. The heterogeneous multi-processor neural network operator scheduling method according to claim 2, characterized in that: The step of obtaining input data for the target neural network model and preprocessing the input data to obtain corresponding data processing results includes: Acquiring input data for the target neural network model, and performing data analysis on the input data to obtain a data analysis result; The input data is divided into blocks according to the data analysis result, and a corresponding target block index table is generated according to the block result of the input data; A corresponding computing resource allocation table is created according to the target block index table, and the input data, the target block index table and the computing resource allocation table are merged to obtain a data processing result.

4. The heterogeneous multi-processor neural network operator scheduling method according to claim 3, characterized in that: The step of dividing the input data into blocks according to the data analysis result and generating a corresponding target block index table according to the block result of the input data includes: Divide the input data into blocks according to the data analysis result, and determine whether there is a non-zero element in each block; If the non-zero element does not exist in the block, the block is marked as a first target block; if the non-zero element exists in the block, the block is marked as a second target block; The marking result is updated to the initial block index table to obtain a target block index table corresponding to the block division result of the input data.

5. The heterogeneous multi-processor neural network operator scheduling method according to claim 4, characterized in that: The step of creating a corresponding computing resource allocation table according to the target block index table includes: Determine whether the number of the first target blocks in the target block index table is greater than a preset number threshold, and distribute the input data based on the determination result to obtain a corresponding distribution result; Creating a corresponding computing resource allocation table according to the allocation result; Wherein, allocating the input data based on the judgment result includes: If the number of the first target blocks is greater than the preset number threshold, allocating the input data to a first preset calculation unit; If the number of the first target blocks is not greater than the preset number threshold, allocating the input data to a second preset calculation unit; The computing capability of the first preset computing unit is smaller than that of the second preset computing unit.

6. The heterogeneous multi-processor neural network operator scheduling method according to claim 3, characterized in that: The creating a data mapping table based on the data processing result and the data flow graph includes: Based on the data processing result, the input and output data of the neural network operators corresponding to the respective computing nodes are determined from the third preset storage space, and a data mapping table is created.

7. The method for scheduling heterogeneous multi-processor neural network operators according to claim 6, characterized in that: The accelerator calculates the input data by scheduling an accelerator operator based on the data flow graph and the data mapping table to obtain output data, including: The accelerator obtains the computing node index based on the data flow graph according to the execution order of each computing node stored in the first preset storage space; The accelerator obtains, from the second preset storage space based on the computing node index, a target operator type corresponding to the computing node in the data flow graph; The accelerator obtains input and output data of the target neural network operator corresponding to the computing node in the data flow graph from the third preset storage space based on the computing node index and the data mapping table; The accelerator calls a corresponding accelerator operator based on the target operator type and the input and output data of the target neural network operator; The accelerator operator determines a target computing unit for computing current input data according to the computing resource allocation table, so that the target computing unit computes the current input data to obtain output data.

8. The method for scheduling heterogeneous multi-processor neural network operators according to claim 7, characterized in that: The target computing unit computes the current input data to obtain output data, including: The target calculation unit calculates the second target block in the current input data based on the target block index table to obtain output data.

9. A heterogeneous multi-processor neural network operator scheduling device, characterized in that: Applied to host-side CPUs and device-side accelerators, including: A neural network model parsing module, used to acquire and parse the target neural network model to obtain a data flow graph, and store the data flow graph in a preset storage space; An input data preprocessing module, used to obtain input data for the target neural network model and preprocess the input data to obtain corresponding data processing results; A data mapping table creation module, used to create a data mapping table based on the data processing result and the data flow graph, and send the data mapping table to the accelerator; An output data calculation module is used for the accelerator to calculate the input data by scheduling the accelerator operator based on the data flow graph and the data mapping table to obtain output data.

10. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, used to execute the computer program to implement the heterogeneous multi-processor neural network operator scheduling method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the heterogeneous multi-processor neural network operator scheduling method as described in any one of claims 1 to 8 is implemented.

Images