Acceleration method and system of AI processor and chip

Abstract

The invention discloses an acceleration method and system of an AI processor and a chip, and belongs to the technical field of electronics. According to the technical scheme of the acceleration method of the AI processor, after a target model which cannot be effectively accelerated by the AI processor is recognized, a first operator which cannot be effectively accelerated in the target model is determined; and determining a target file corresponding to the first operator, and synchronizing the target file to an operator acceleration library of an eFPGA of the AI processor so as to accelerate the first operator through the eFPGA. The adaptability and the processing energy efficiency of the AI processor can be improved.

Description

Technical Field

[0001] The present application relates to the field of electronic technology, and in particular to an acceleration method, system and chip for an AI processor. Background Art

[0002] In recent years, the field of deep learning has made important progress in intelligent perception. In many aspects such as speech, image and video processing, the efficiency and effectiveness of artificial intelligence processing have surpassed that of humans. With the rapid development of large models in the past two years, artificial intelligence has also made significant progress in language understanding and processing.

[0003] Because deep learning algorithms have the characteristics of data independence and parallel computing, and the computing type is intensive, they are very suitable for hardware acceleration, which has promoted the vigorous development of AI (Artificial Intelligence) processors.

[0004] However, with the continuous emergence of new AI technologies, AI models and operators are changing with each passing day, which has a great impact on DSA's customized AI processors. This is manifested in the inability to effectively accelerate the execution of new models and new operators, affecting the effective life and competitiveness of AI processors. For model operators that cannot be effectively processed or are not supported by AI processors, they need to interact with the CPU (Central Processing Unit / Processor) for processing, or even can only be executed by the CPU, which is extremely inefficient and does not meet the needs of scenarios with high real-time requirements. In other words, for new models and new operators in new AI technologies, the adaptability and processing energy efficiency of AI processors are low, which has become a problem that needs to be solved urgently. Summary of the invention

[0005] The main purpose of this application is to provide an acceleration method, system and chip for an AI processor, aiming to solve the technical problems of the adaptability and low processing efficiency of current AI processors.

[0006] To achieve the above objectives, an embodiment of the present application provides an acceleration method for an AI processor, the method comprising:

[0007] After identifying a target model that cannot be effectively accelerated by the AI ​​processor, determining a first operator in the target model that cannot be effectively accelerated;

[0008] Cooperating with a microprocessor and an ASIC circuit of the AI ​​processor to execute the first operator;

[0009] Determine a target file corresponding to the first operator, and synchronize the target file to an operator acceleration library of the eFPGA of the AI ​​processor to accelerate the first operator through the eFPGA.

[0010] In addition, to achieve the above purpose, the embodiment of the present application also provides an acceleration system for an AI processor, including:

[0011] A compiling module, configured to, after identifying a target model that cannot be effectively accelerated by the AI ​​processor, determine a first operator in the target model that cannot be effectively accelerated;

[0012] An acceleration module is configured to determine a target file corresponding to the first operator, and synchronize the target file to an operator acceleration library of the eFPGA of the AI ​​processor to accelerate the first operator through the eFPGA.

[0013] In addition, an embodiment of the present application also provides a chip, including: a memory, a processor, and an acceleration program stored in the memory and executable on the processor, wherein when the acceleration program is executed by the processor, the acceleration method of the AI ​​processor as described above is implemented.

[0014] In addition, an embodiment of the present application also provides a computer-readable storage medium, on which an acceleration program for an AI processor is stored. When the acceleration program for the AI ​​processor is executed by a processor, the steps of the acceleration method for the AI ​​processor as described above are implemented.

[0015] The embodiments of the present application disclose an acceleration method, system and chip for an AI processor. The technical solution of the acceleration method for the AI ​​processor is to, after identifying a target model that the AI ​​processor cannot effectively accelerate, determine a first operator that cannot be effectively accelerated in the target model, then determine a target file corresponding to the first operator, and synchronize the target file to an operator acceleration library of an eFPGA (Embedded FPGA, embedded programmable gate array) of the AI ​​processor to accelerate the first operator through the eFPGA, thereby using the eFPGA to accelerate operators that cannot be effectively processed by microprocessors and ASIC circuits, effectively solving the problems of low adaptability and processing efficiency of the AI ​​processor in dealing with new AI technologies, new models and new operators, and improving the effective market life and competitiveness of the AI ​​processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0017] Figure 1 This is a flowchart of the first embodiment of the acceleration method of the AI ​​processor of the present application;

[0018] Figure 2 This is a flow chart of the second embodiment of the acceleration method of the AI ​​processor of the present application;

[0019] Figure 3 This is a schematic diagram of the structure of the AI ​​processor of the first embodiment of the present application;

[0020] Figure 4 This is a schematic diagram of the structure of the AI ​​processor according to the second embodiment of the present application;

[0021] Figure 5 This is a schematic diagram of the structure of the AI ​​processor of the third embodiment of the present application;

[0022] Figure 6 This is a flowchart of an AI processor according to a specific embodiment of the present application;

[0023] Figure 7 This is a schematic diagram of the functional modules of the acceleration system of the AI ​​processor according to an embodiment of the present application;

[0024] Figure 8 This is a schematic diagram of the hardware structure of the chip involved in the embodiment of the present application.

[0025] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0026] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] It should be noted that all directional indications in the embodiments of the present application (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0029] In this application, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0030] In addition, in this application, descriptions such as "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0031] Because deep learning algorithms have the characteristics of data independence and parallel computing, and the computing type is intensive, they are very suitable for hardware acceleration, which has promoted the vigorous development of AI (Artificial Intelligence) processors.

[0032] However, with the continuous emergence of new AI technologies, AI models and operators are changing with each passing day, which has a great impact on DSA's customized AI processors. This is manifested in the inability to effectively accelerate the execution of new models and new operators, affecting the effective life and competitiveness of AI processors. For model operators that cannot be effectively processed or are not supported by AI processors, they need to interact with the CPU (Central Processing Unit / Processor) for processing, or even can only be executed by the CPU, which is extremely inefficient and does not meet the needs of scenarios with high real-time requirements. In other words, for new models and new operators in new AI technologies, the adaptability and processing energy efficiency of AI processors are low, which has become a problem that needs to be solved urgently.

[0033] In order to improve the competitiveness of AI processors, related technologies integrate multiple multi-core microprocessors in AI processors. New operators that cannot be processed are implemented by microprocessors, which is extremely inefficient. In addition, the large number of integrated microprocessors will lead to high costs for AI processors. Some even need to be processed by the host CPU outside the AI ​​processor. The AI ​​processor needs to interact a lot with the host, which is very inefficient.

[0034] Based on this, the embodiment of the present application provides an acceleration method for an AI processor, referring to Figure 1 , Figure 1 The following is a flow chart of an embodiment of an acceleration method for an AI processor of the present application. In this embodiment, the acceleration method for the AI ​​processor includes:

[0035] Step S10, after identifying a target model that cannot be effectively accelerated by the AI ​​processor, determining a first operator in the target model that cannot be effectively accelerated;

[0036] It is known to those skilled in the art that an AI (Artificial Intelligence) processor is an integrated circuit chip designed specifically for artificial intelligence applications. It can accelerate the operation and processing of artificial intelligence algorithms at the hardware level, improving the efficiency and performance of the algorithms. AI processors generally include multiple core components such as processors, memory, and accelerators, which can realize a variety of artificial intelligence applications such as image recognition, natural language processing, speech recognition, machine translation, or autonomous driving.

[0037] In this embodiment, the target model refers to a model that the AI ​​processor cannot effectively accelerate. It is understandable that the model to be deployed by the user can be automatically deployed to the AI ​​processor for acceleration after passing through the compiler. Among them, the model to be deployed by the user can be a model trained based on frameworks such as Tensorflow / Caffe / Pytorch / PP Feijiang. It should be noted that the various processing engines of the AI ​​processor (which may include engines of eFPGA, microprocessor and ASIC circuit) are executed asynchronously and in parallel under the control of the compilation instructions to complete the acceleration of the entire model, such as Figure 2 When the model to be deployed by the user cannot be effectively accelerated by each processing engine of the AI ​​processor, it is determined that the model to be deployed by the user is a target model that cannot be effectively accelerated by the AI ​​processor.

[0038] In one embodiment, when the AI ​​processor is accelerating the model to be deployed by the user, if the engine resource utilization rate of the AI ​​processor is less than a preset ratio threshold, the model is identified as a target model that cannot be effectively accelerated. In another embodiment, if each processing engine of the AI ​​processor fails to accelerate the model to be deployed by the user within a predetermined time period, the model is identified as a target model that cannot be effectively accelerated. In yet another embodiment, the target model that cannot be effectively accelerated may be a manually marked model, and the system may identify whether the current model is a target model that cannot be effectively accelerated by the AI ​​processor by determining whether the model carries the identification information of the manual mark. This embodiment does not specifically limit this.

[0039] Exemplarily, the step of determining the first operator in the target model that cannot be effectively accelerated includes:

[0040] Step A10, parsing the target model through a compiler to obtain multiple operators to be accelerated;

[0041] Step A20: Determine, from the operators to be accelerated, an operator that cannot be effectively run on the engine resources of the AI ​​processor as the first operator that cannot be effectively accelerated in the target model.

[0042] In one embodiment, when the AI ​​processor is accelerating the operator to be accelerated, if the engine resource utilization rate of the AI ​​processor is less than a preset ratio threshold, it is identified that the operator to be accelerated cannot be effectively run on the engine resources of the AI ​​processor, that is, the operator to be accelerated is the first operator that cannot be effectively accelerated. In another embodiment, each processing engine of the AI ​​processor fails to complete the acceleration of the operator to be accelerated within a predetermined time period, and it is identified that the operator to be accelerated cannot be effectively run on the engine resources of the AI ​​processor, that is, the operator to be accelerated is the first operator that cannot be effectively accelerated. In yet another embodiment, the operator to be accelerated that cannot be effectively accelerated may be an operator that is manually marked, and the system may identify whether the operator to be accelerated cannot be effectively run on the engine resources of the AI ​​processor (that is, determine whether the operator to be accelerated is the first operator that the AI ​​processor cannot effectively run) by determining whether the operator to be accelerated carries the marking information of the manual marking. This embodiment does not make specific limitations on this.

[0043] It is understandable that the AI ​​processor can be made into a PCIe (peripheral component interconnect express) card and plugged into the server, and the server can be used as an AI server. Multiple PCIe cards containing AI processors can be plugged into the server to enhance the processing power of the AI ​​server. AI servers can be cascaded to form an AI server cluster, which can meet various applications in cloud and edge scenarios.

[0044] This embodiment parses the target model through a compiler to obtain multiple operators to be accelerated, and then based on the operators in each operator that cannot be effectively run on the engine resources of the AI ​​processor, it is convenient to accurately obtain the first operator in the target model that cannot be effectively accelerated.

[0045] Step S20, determine the target file corresponding to the first operator, and synchronize the target file to the operator acceleration library of the eFPGA of the AI ​​processor to accelerate the first operator through the eFPGA.

[0046] In this embodiment, by synchronizing the target file corresponding to the first operator to the operator acceleration library of the eFPGA of the AI ​​processor, the eFPGA can subsequently accelerate the first operator by loading the target file in the operator acceleration library.

[0047] It should be noted that the acceleration of the first operator through eFPGA may include four situations: the first situation is to collaboratively execute the first operator through eFPGA (Embedded FPGA, embedded programmable gate array) and ASIC circuit. The second situation is to collaboratively execute the first operator through eFPGA and microprocessor. The third situation is to collaboratively execute the first operator through eFPGA, microprocessor and ASIC circuit. The fourth situation is to execute the first operator only through eFPGA.

[0048] The embodiments of the present application disclose an acceleration method, system and chip for an AI processor. The technical solution of the acceleration method for the AI ​​processor is to, after identifying a target model that the AI ​​processor cannot effectively accelerate, determine a first operator that cannot be effectively accelerated in the target model, then determine a target file corresponding to the first operator, and synchronize the target file to an operator acceleration library of an eFPGA (Embedded FPGA, embedded programmable gate array) of the AI ​​processor to accelerate the first operator through the eFPGA, thereby using the eFPGA to accelerate operators that cannot be effectively processed by microprocessors and ASIC circuits, effectively solving the problems of low adaptability and processing efficiency of the AI ​​processor in dealing with new AI technologies, new models and new operators, and improving the effective market life and competitiveness of the AI ​​processor.

[0049] In a possible implementation manner, after step S10, the step of determining the first operator in the target model that cannot be effectively accelerated, the method further includes:

[0050] Step B10, cooperatively executing the first operator through the microprocessor and the ASIC circuit of the AI ​​processor;

[0051] Those skilled in the art will appreciate that an ASIC (Application Specific Integrated Circuit) circuit refers to a type of integrated circuit relative to a general-purpose integrated circuit, and is an integrated circuit designed for a certain application field or a special requirement of a certain customer.

[0052] In this embodiment, the collaborative execution of the first operator by the microprocessor and the ASIC circuit of the AI ​​processor means: by a preset scheduling algorithm, the engine resource space of the microprocessor and the ASIC circuit is comprehensively considered, and the first operator is scheduled to the microprocessor and / or the ASIC circuit for operation. Wherein, exemplarily, the compiler allocates the first operator to the microprocessor for execution (when the processing performance of the microprocessor alone is better than that of the microprocessor and / or the ASIC circuit, for example, when the engine resource space of the ASIC circuit is insufficient) through the preset scheduling algorithm, combined with the engine resource space of the microprocessor and the ASIC circuit, or allocates the first operator to the ASIC circuit for execution (when the processing performance of the ASIC circuit alone is better than that of the ASIC circuit and / or the microprocessor, for example, when the engine resource space of the microprocessor is insufficient), or allocates the first operator to the ASIC circuit for execution (when the processing performance of the ASIC circuit alone is better than that of the ASIC circuit and / or the microprocessor, for example, when the engine resource space of the microprocessor is insufficient), or when the first operator is allocated to the ASIC circuit and the microprocessor for joint execution (when the performance of the ASIC circuit and the microprocessor is better than that of the ASIC circuit or the microprocessor alone, for example, when the ASIC circuit and the microprocessor both have a certain engine resource space, and the joint processing of the two can exert the maximum processing efficiency of processing the first operator).

[0053] It is worth mentioning that for the operator that the microprocessor and dedicated ASIC circuit of the AI ​​processor cannot effectively accelerate, and there is no target file corresponding to the operator in the operator acceleration library (that is, the operator is the first operator). In order not to affect the user experience, the embodiment of the present application first performs collaborative execution of the first operator through the microprocessor and ASIC circuit of the AI ​​processor, even if the acceleration performance is not optimal, and then determines the target file corresponding to the first operator, and synchronizes the target file to the eFPGA of the AI ​​processor. Specifically, the compiler can automatically notify the rear of the form of the first operator, and the rear can use programmable logic to implement the first operator in the eFPGA, and add the target file to the operator acceleration library, thereby upgrading the eFPGA program through online upgrades, so that the AI ​​processor has the best processing performance for the new operator, and provides the best performance in a way that minimizes the impact on users, further improving the adaptability and processing energy efficiency of the AI ​​processor to new AI technologies, new models, and new operators.

[0054] Exemplarily, the method further includes:

[0055] Step C10, after identifying the network model that can be effectively accelerated by the AI ​​processor, all operators of the network model are collaboratively executed through the microprocessor, ASIC circuit and eFPGA of the AI ​​processor.

[0056] In this embodiment, the collaborative execution of the first operator through the microprocessor, ASIC circuit and eFPGA of the AI ​​processor means: through a preset scheduling algorithm, taking into account the engine resource space of the microprocessor, ASIC circuit and eFPG, the first operator is scheduled to at least one of the microprocessor, ASIC circuit and eFPGA for operation.

[0057] In this embodiment, the network model refers to a model that can be effectively accelerated by the AI ​​processor. It can be understood that the model to be deployed by the user can be automatically deployed to the AI ​​processor for acceleration after passing through the compiler. Among them, the model to be deployed by the user can be a model trained based on a framework such as Tensorflow / Caffe / Pytorch / PP Feijiang, such as Figure 2 As shown. It should be noted that the processing engines of the AI ​​processor (which may include engines of eFPGA, microprocessor and ASIC circuit) are executed asynchronously and in parallel under the control of the compilation instructions to complete the acceleration of the entire model. If the model to be deployed by the user can be effectively accelerated by the processing engines of the AI ​​processor, the model to be deployed by the user is determined to be a network model that can be effectively accelerated by the AI ​​processor.

[0058] After identifying a network model that can be effectively accelerated by the AI ​​processor, this embodiment uses the AI ​​processor's microprocessor, ASIC circuit, and eFPGA to collaboratively execute all operators of the network model, thereby making full use of all engine resources of the AI ​​processor, accelerating the model to be deployed, improving computing efficiency, and improving the computing power of the AI ​​processor, so that the AI ​​processor can adapt to different scenarios and AI algorithms, improving its flexibility, and thereby improving the adaptability and processing energy efficiency of the AI ​​processor in dealing with new AI technologies, new models, and new operators.

[0059] In a possible implementation manner, after the step of parsing the target model through a compiler to obtain a plurality of operators to be accelerated, the method further includes:

[0060] Step D10, mapping each operator to be accelerated to the engine resources for execution processing by the compiler based on the engine resources in the AI ​​processor in combination with a preset scheduling algorithm;

[0061] In this embodiment, the preset scheduling algorithm has been studied in depth by those skilled in the art, and will not be described here. In addition, this embodiment does not specifically limit the algorithm type of the preset scheduling algorithm, and is based on the ability to map the model operator to be deployed to the engine resources in the AI ​​processor for execution processing.

[0062] Step D20, taking the operator to be accelerated that does not include the eFPGA engine in the mapped engine resources as the second operator, and identifying whether the second operator can be effectively accelerated by the microprocessor and / or ASIC circuit of the AI ​​processor;

[0063] It is known to those skilled in the art that when the compiler performs execution processing on the model operator to be deployed based on the engine resources in the AI ​​processor, it will determine which engine resource can effectively accelerate the operator (i.e., the model operator), such as which one of the eFPGA, microprocessor, and ASIC circuit, or which combination of the eFPGA, microprocessor, and ASIC circuit can effectively accelerate the operator. When it is determined that the eFPGA can effectively accelerate a certain model operator A, the model operator A will be given priority to the eFPGA for execution processing. For another example, when it is determined that the eFPGA cannot effectively accelerate a certain model operator B, the model operator B will generally not be given priority to the eFPGA engine for effective acceleration, that is, the engine resources mapped by the model operator B often do not include the eFPGA engine. If the engine resources mapped by the model operator B do not include the eFPGA engine, then the model operator B is the "second operator" referred to above, and at this time it is identified whether the second operator can be effectively accelerated by the microprocessor and / or ASIC circuit.

[0064] Step D30: When it is identified that the second operator cannot be effectively accelerated, the second operator is used as an operator among the operators to be accelerated that cannot be effectively run on the engine resources of the AI ​​processor.

[0065] In this embodiment, if it is identified that the second operator cannot be effectively accelerated by the microprocessor and / or ASIC circuit, it means that the second operator cannot be effectively accelerated by all engine resources of the AI ​​processor. At this time, the second operator is the operator among all operators that cannot be effectively run on the engine resources of the AI ​​processor.

[0066] In this embodiment, the compiler maps each operator to the engine resources for execution processing based on the engine resources in the AI ​​processor in combination with a preset scheduling algorithm, and uses the operator to be accelerated that does not include the eFPGA engine in the mapped engine resources as the second operator, and then identifies whether the second operator can be effectively accelerated by the microprocessor and / or ASIC circuit. When it is identified that the second operator cannot be effectively accelerated, the second operator is used as the operator that cannot be effectively run on the engine resources of the AI ​​processor among the operators, thereby facilitating further accurate identification of the first operator that cannot be effectively accelerated in the target model.

[0067] Furthermore, in a possible implementation, the engine resources include the eFPGA, the microprocessor, and the engine of the ASIC circuit, and the step of mapping each operator to be accelerated to the engine resources for execution processing by the compiler based on the engine resources in the AI ​​processor in combination with a preset scheduling algorithm includes:

[0068] Step E10, identifying engine resources in the AI ​​processor through a compiler to determine whether the engine of the ASIC circuit can accelerate each operator to be accelerated;

[0069] Step E20, taking the operator to be accelerated that cannot be accelerated by the engine of the ASIC circuit as the third operator, and determining whether the engine of the eFPGA can accelerate the third operator;

[0070] Step E30, mapping the third operator that cannot be accelerated by the engine of the eFPGA to the engine of the microprocessor and the ASIC circuit through a preset scheduling algorithm for collaborative execution

[0071] It should be noted that the engine resources in the AI ​​processor include the eFPGA engine (the eFPGA engine is the "eFPGA engine" mentioned above), the microprocessor engine, and the ASIC circuit engine.

[0072] In this embodiment, if the operator to be accelerated can be effectively accelerated by the ASIC circuit, the ASIC circuit is used for effective acceleration first. The remaining operators need to determine whether the eFPGA engine resources (operator acceleration library) can be accelerated. The operators that cannot be accelerated by the eFPGA engine among the remaining operators are the second operators.

[0073] In this embodiment, if an operator C needs to be accelerated by the eFPGA engine, the eFPGA operator acceleration library needs to store the target file corresponding to the operator C. For ease of description, the target file corresponding to the operator C is referred to as the target file c. The eFPGA engine can then load the target file c from the operator acceleration library to accelerate the operator C. In addition, a special case needs to be considered, that is, even if the target file c exists in the operator acceleration library, if the resource space of the eFPGA engine is fully occupied, it is impossible to free up sufficient engine resources (i.e., the computing power resources of the eFPGA) to load the target file c into the operator acceleration library to accelerate the operator C. At this time, effective acceleration of the operator C cannot be achieved. Therefore, in this embodiment, based on the identification and processing of the eFPGA operator acceleration library by the compiler (i.e., identifying whether there is a corresponding target file for the eFPGA operator acceleration library), the eFPGA engine is also combined to identify whether the eFPGA engine can accelerate each operator (i.e., whether the computing power resources of the eFPGA engine are sufficient) to improve the accuracy of identifying whether the current model operator can be accelerated by the eFPGA engine.

[0074] Further, in a possible implementation manner, after the step of synchronizing the target file to the operator acceleration library of the eFPGA of the AI ​​processor, the method further includes:

[0075] Step F10, sending notification information to the compiler, wherein the notification information is used to trigger the compiler to identify and process the engine resources in the AI ​​processor.

[0076] In this embodiment, after the target file is added to the operator acceleration library (that is, after the eFPGA program is upgraded), the eFPGA engine now has the ability to execute the first operator (the target file added to the operator acceleration library is the target file corresponding to the first operator), or in other words, the eFPGA has the ability to accelerate the first operator. At this time, after the step of synchronizing the target file to the operator acceleration library of the eFPGA of the AI ​​processor, this embodiment also sends a notification message to the compiler, and the compiler responds to the notification message and executes again: the step of identifying the operator acceleration library of the eFPGA by the compiler, so that the compiler can re-identify and identify the target file of the first operator in the operator acceleration library of the eFPGA, that is, identify the eFPGA with The first operator is accelerated by the AI ​​processor. At this time, the original operator acceleration logic of "collaborative execution of the first operator through the microprocessor and ASIC circuit of the AI ​​processor" is converted to the operator acceleration logic of "collaborative execution through the microprocessor, ASIC circuit and eFPGA engine". Since the eFPGA engine is added for acceleration this time, this embodiment makes full use of all engine resources of the AI ​​processor to accelerate the model to be deployed, improve computing efficiency, and improve the computing power of the AI ​​processor. This embodiment upgrades the eFPGA program through online upgrades to optimize the processing performance of the AI ​​processor for the new operator, and provides the best performance in a way that minimizes the impact on users, thereby improving the adaptability and processing energy efficiency of the AI ​​processor to new AI technologies, new models, and new operators.

[0077] This embodiment notifies the compiler to re-evaluate the processing engine resources of the AI ​​processor after the circuit acceleration library of the eFPGA is synchronized. The eFPGA reloads the appropriate circuit through online download, and the compiler re-divides and distributes the tasks to be processed by each processing engine to achieve optimized performance and improve the robustness of the AI ​​processor.

[0078] In an practicable manner, the step of determining the target file corresponding to the first operator includes:

[0079] Step G10, outputting the information of the first operator;

[0080] Step G20, obtaining an acceleration unit execution file corresponding to the first operator input based on the output prompt, wherein the acceleration unit execution file is executable by the eFPGA;

[0081] Step G30, using the acceleration unit execution file as the target file corresponding to the first operator.

[0082] This embodiment outputs the information of the first operator, thereby automatically notifying the back-end (i.e., the back-end engineer) of the form of the first operator, so that the back-end can implement the first operator in the eFPGA using programmable logic, and then obtain the acceleration unit execution file corresponding to the first operator input based on the output prompt (the acceleration unit execution file is made by the back-end based on the form of the first operator), wherein the acceleration unit execution file is executable by the eFPGA, and then the acceleration unit execution file is used as the target file corresponding to the first operator, so as to facilitate adding the target file to the operator acceleration library, effectively realize the upgrade of the eFPGA program based on the online upgrade method, and ensure the optimal processing performance of the AI ​​processor for the new operator.

[0083] In order to help understand the technical concept or technical principle of the embodiments of the present application, a specific embodiment 1 is listed:

[0084] This specific embodiment provides an AI processor, which contains a microprocessor (ARM, RISC-V or self-developed), a dedicated ASIC circuit and an eFPGA, and organically connects the microprocessor, the ASIC circuit and the eFPGA through internal buses such as NOC, AXI4, AHB, etc. The eFPGA is used to accelerate operators that cannot be effectively processed by the microprocessor and the ASIC circuit, effectively solving the adaptability and processing energy efficiency problems of new AI technologies, new models, and new operators, and improving the market effective life and competitiveness of the AI ​​processor.

[0085] The connection diagram between ASIC circuit and eFPGA is as follows Figure 3 As shown. The control management channel connecting the microprocessor in the ASIC circuit to the eFPGA is responsible for the loading and updating operations and reset control management of the eFPGA target program. The dedicated ASIC circuit and the eFPGA have a synchronous control channel and a data exchange channel. The synchronous control channel is responsible for the synchronous information transmission of the collaborative processing between the microprocessor, the dedicated ASIC circuit and the eFPGA, and the data exchange channel is responsible for transmitting the collaborative data transmission between the microprocessor, the dedicated ASIC circuit and the eFPGA.

[0086] The connection diagram between the dedicated ASIC circuit and the eFPGA is shown in the figure Figure 4 As shown, the dedicated ASIC circuit has efficient processing units customized for AI computing, such as matrix computing engines, vector processing engines, and data movement and transformation engines. The engines exchange data through buffers (i.e., Buffers) and drive computing through data flows. The dedicated ASIC circuit and the eFPGA have a synchronous control channel and a data exchange channel. The synchronous control channel is responsible for synchronous information transmission for collaborative processing between the microprocessor, the dedicated ASIC circuit, and the eFPGA, and the data exchange channel is responsible for transmitting collaborative data transmission between the microprocessor, the dedicated ASIC circuit, and the eFPGA.

[0087] Among them, the microprocessor can be composed of ARM, RISC-V or self-developed microarchitecture. The AISC circuit includes a matrix (tensor) processing engine (Tensor Engine), a vector processing engine (Vector Engine) and a data transfer transformation engine (DataSwitch Engine). The microprocessor is used for instruction distribution and control, the Tensor Engine is used to process matrix (tensor) operations, and the Vector Engine is used to process vector operations. The main function of the data transfer transformation engine is data deformation and conversion and the realization of efficient data transfer between multiple buffers and between buffers and DDR / HBM.

[0088] Specifically, the matrix (tensor) computing engine, vector computing engine, data movement and transformation engine, and eFPGA processing unit need to receive instructions distributed by the microprocessor engine to complete various predefined instruction operations. The microprocessor provides target file update channels and control channels for the eFPGA, can reset the eFPGA or ASIC circuit, and can also process special and complex operations that the ASIC circuit and eFPGA cannot handle.

[0089] There are multiple engines for parallel acceleration inside the ASIC circuit, including the matrix (tensor) calculation engine, the vector calculation engine, and the data movement and transformation engine. The matrix (tensor) calculation engine takes out activations and weights from the Factory Buffer and the Weight Buffer respectively to perform matrix multiplication operations, and stores the calculation results in the Middle Buffer for partial sum accumulation or output. The vector calculation engine is responsible for completing the processing of maxpooling, average pooling, relu, prelu, normalization, scale, quantization and other links except convolution.

[0090] eFPGA is used to accelerate operators that cannot be effectively processed by microprocessors and ASIC circuits. The goal of eFPGA circuit acceleration is to classify and abstract operator units in models such as CNN, RNN, and Transformer that cannot be effectively processed by microprocessors and ASIC circuits. After these operators are abstracted and implemented with eFPGA, multiple target files are generated and placed in the operator acceleration library. According to the actual situation of the deployed model, the target file of the corresponding operator is loaded into eFPGA for acceleration.

[0091] The acceleration library file of eFPGA can be stored on the server and transmitted to the DDR (Double Data Rate) / HBM (High Bandwidth Memory) memory of the AI ​​processor through the PCIe interface (i.e., bus interface). The microprocessor reads the target file from the DDR / HBM and loads the target file into the eFPGA through the GPIO (General-Purpose Input / Output Ports) interface to simulate the JTAG (Joint Test Action Group) interface timing. According to actual needs, the microprocessor can reset the eFPGA through the reset control channel. The eFPGA has an Operator Cmd Queue inside to store the instructions of the acceleration operator implemented by the eFPGA, and the Data Buffer is used to store the data to be processed by the Operator_1Function...Operator_n Function operator acceleration unit and the results returned after processing. Operator_1Function...Operator_nFunction operators can be cascaded to implement more complex operators. Operator_1Function...Operator_n Function operators can be updated online to operators with other functions according to the acceleration requirements of the current model. Data Buffer can be set to multiple ping-pong Buffers as needed to reduce Buffer access conflicts.

[0092] The principle block diagram of the AI ​​processor is as follows Figure 5 The microprocessor, eFPGA, and different processing engines exchange data through buffers, and drive calculations through data flows to improve operating efficiency.

[0093] In this embodiment, the microprocessor controls the loading and resetting of the eFPGA. The eFPGA acceleration circuit and the acceleration engines (matrix / tensor processing engine, vector processor, and scalar processor) in the ASIC circuit exchange data and work together through ping-pongBuffer. The compiler splits the model into micro-units that these processing engines can process, and the microprocessor sends it to each processing engine for collaborative and efficient processing.

[0094] In view of the rapid development of new AI technologies, new models, and new operators, for new operators that cannot be effectively accelerated, we can continuously design and implement new operator circuits, generate target files, and synchronize them to the acceleration library. This can effectively deal with the low processing efficiency of DSA (Domain Specific Architecture) customized AI chips caused by the endless emergence of new technologies, new models, and new operators.

[0095] In this embodiment, for operators that cannot be effectively processed by microprocessors and ASIC circuits, eFPGA (embedded programmable logic) is used to make acceleration modules, generate target files, and integrate these operator acceleration modules into an operator acceleration library.

[0096] For models processed by the AI ​​processor, the compiler first parses the model to confirm the operators that need to be accelerated. It also determines whether the current AI processor resources can effectively process the model. The processor resources include the microprocessor, ASIC circuit, and the operator acceleration library corresponding to the eFPGA. If the model can be effectively processed, the target file of the corresponding operator module is loaded into the eFPGA, and the microprocessor, ASIC circuit, and eFPGA work together effectively to perform efficient training or reasoning tasks of the model. Otherwise, the compiler points out that the current resources cannot effectively process the operator, and the corresponding operator can be added through design and implemented with eFPGA resources. After the design is completed, the target file is generated and synchronized to the operator acceleration library. If the operator is involved in the subsequent processing, the operator target file can be directly loaded for effective acceleration.

[0097] For operators that cannot be effectively accelerated in subsequent AI new technologies, new models, and new operators, eFPGA resources can be used to accelerate them, and the target files after implementation can be placed in the operator acceleration library. Because eFPGA is programmable, these operators implemented with eFPGA can also be iterated and optimized and upgraded in the future.

[0098] It is worth mentioning that for the equipment used in the field, the operators parsed by the user model through the compiler cannot be effectively accelerated by the microprocessor and dedicated ASIC circuit, and there is no corresponding operator acceleration target file in the acceleration library. In order not to affect the user, the compiler first considers accelerating the operator with the engine in the microprocessor and dedicated ASIC circuit (even if the performance is not optimal). Then notify the rear that the operator cannot be effectively accelerated. The rear can use programmable logic to implement the operator in the eFPGA, add the target file to the acceleration library, and upgrade the eFPGA program through online upgrade to optimize the processing performance of the operator, providing the best performance with minimal impact on users.

[0099] This specific embodiment proposes an AI processor based on the adaptability and processing energy efficiency of DSA customized AI chips to cope with new AI technologies, new models, and new operators. The processor contains a microprocessor (ARM, RISC-V or self-developed), a dedicated ASIC circuit, and an eFPGA. The microprocessor, ASIC circuit, and eFPGA are organically connected through internal buses such as NOC, AXI4, AHB, etc. The eFPGA is used to accelerate operators that cannot be effectively processed by the microprocessor and ASIC circuit. Through the classification and abstraction of new AI technologies, new models, and new operators, the new operators that appear are implemented with circuits and loaded into the eFPGA, which effectively solves the adaptability and processing energy efficiency of DSA customized AI chips to cope with new AI technologies, new models, and new operators, and improves the market effective life and competitiveness of AI processors.

[0100] It should be noted that although the above-mentioned specific embodiments show many details, they are only used to help understand the technical concepts or technical principles of the embodiments of the present application, and do not constitute a limitation on the present application. More simple transformations based on the technical concepts should all be within the scope of protection of the present application.

[0101] In a possible implementation, the method further includes:

[0102] Step H10, when it is identified that there are multiple first operators in the target model and the engine based on the eFPGA cannot execute and process each of the first operators at the same time, determine the priority of each of the first operators;

[0103] Step H20, fill each first operator into the resource space of the eFPGA engine for execution processing in order of priority from high to low, and the remaining other first operators after the resource space is filled are collaboratively executed through the microprocessor of the AI ​​processor, the ASIC circuit and the eFPGA engine.

[0104] Exemplarily, the step of determining the priority of each first operator includes:

[0105] Step I10, identifying the influence weight of each of the first operators on the task processing performance of the network model;

[0106] Step I20: determining the priority of each of the first operators according to the impact weight, wherein the greater the impact weight of the task processing performance, the higher the priority.

[0107] In this embodiment, if in the model of the AI ​​processor, the compiler parses out multiple operators that cannot be effectively processed, the operators that cannot be effectively processed are sorted according to priority, and the operator with the greatest impact on the overall performance has the highest priority, and the operator with the least impact on the overall performance has the lowest priority. If multiple acceleration library operators are to be used, according to the resources of the eFPGA, if the resources are sufficient, the design containing these operators is integrated to generate a new target file and loaded into the eFPGA. If the resources are insufficient, the operators are implemented in order from high priority to low priority, and some high-priority operators are integrated to generate a target file and loaded into the eFPGA.

[0108] In this embodiment, when it is identified that there are multiple first operators in the target model and the engine based on the eFPGA cannot execute the first operators at the same time, the priority of each first operator is determined, and then each first operator is filled into the resource space of the eFPGA engine for execution in descending order of priority of each first operator, and the remaining first operators after the resource space is filled are collaboratively executed by the microprocessor, the ASIC circuit and the eFPGA engine, so that this embodiment not only identifies whether the eFPGA has the ability to accelerate each operator (that is, identifies whether there is a corresponding target file for the operator acceleration library of the eFPGA), but also comprehensively considers the computing power resource space of the eFPGA engine, and on this basis, combined with the consideration of the influence weight of each first operator on the task processing performance of the network model when the computing power resources of the eFPGA are limited, determines the priority of the eFPGA to execute each first operator, so that the overall processing performance of the AI ​​processor for the network model deployed by the user is optimized, so that the AI ​​processor can adapt to different scenarios and AI algorithms, improve its flexibility, and further improve the adaptability and processing energy efficiency of the AI ​​processor to new AI technologies, new models, and new operators.

[0109] In order to further help understand the technical concept or technical principle of the embodiment of the present application, a specific embodiment 2 is listed. The AI ​​processor workflow diagram of this specific embodiment is as follows: Figure 6 As shown:

[0110] 1. For the models that users need to deploy, the software compiler parses the models and integrates and optimizes them, including operator fusion and parallel optimization.

[0111] 2. After parsing the model, the compiler splits the model into micro-units that can be processed by these processing engines based on the current engine resources of the AI ​​processor, including the microprocessor, the matrix (tensor) processing engine in the ASIC circuit, the vector processing engine and the data movement and transformation engine, and the circuit acceleration library processing unit of the eFPGA.

[0112] 3. If after parsing the model, there are new operators that cannot be effectively processed by the current processing engine resources, in order not to affect user deployment, the compiler should first use one or more of the processing engines such as microprocessors, ASIC circuits, and acceleration units in the eFPGA circuit acceleration library to collaboratively process the new operators that cannot be effectively processed, first meet user deployment requirements, and then optimize performance through online upgrades.

[0113] 4. The new operators that cannot be effectively processed are transmitted to the backend by printing information. The developer analyzes the new operators that cannot be effectively processed, implements the operators with eFPGA, and adds the target files of the implemented operators to the circuit acceleration library of eFPGA.

[0114] 5. After the eFPGA circuit acceleration library is synchronized, the compiler is notified to re-evaluate the processing engine resources of the AI ​​processor. The eFPGA reloads the appropriate circuit through online download, and the compiler re-divides and allocates the tasks of each processing engine to achieve the goal of optimizing performance and improving the competitiveness of the AI ​​processor.

[0115] Specifically, the AI ​​processor is used to accelerate the performance of deep learning models, provide powerful computing resources, speed up deployment and improve user experience. The current deep learning models mainly include CNN, RNN and Transformer models. This embodiment takes the basic model Transformer of chatGPT as an example to introduce in detail:

[0116] First, the software compiler parses the Transformer basic model and finds out that deploying this model requires efficient implementation of operators such as MatMul, Scale, Softmax, Layer Normalization, and Concat.

[0117] Matmul is a matrix multiplication operation that can be efficiently implemented using the matrix (tensor) computing engine in the ASIC circuit. Scale operation can be efficiently implemented using the vector computing engine in the ASIC circuit. Concat can be efficiently implemented using the data movement and transformation engine in the ASIC circuit.

[0118] In the CNN model, Softmax is usually in the last layer of the model, and the amount of data that needs to be calculated is small. Using a microprocessor or sending it to the Host CPU through the PCIe interface is not a performance bottleneck. However, in the Transformer model, the Softmax operator is used in each layer, and the amount of data that needs to be calculated in each layer is very large. If it is sent to the Host CPU for processing by a microprocessor or through the PCIe interface, it will seriously affect the processing performance of the AI ​​processor, thereby affecting the competitiveness of the AI ​​processor. If there is a target file for implementing Softmax operator acceleration in the eFPGA acceleration library file, the target file for accelerating Softmax can be loaded into the eFPGA, and eFPGA can be used for acceleration when performing Softmax operations. If there is no target file for Softmax operator acceleration in the acceleration library, the compiler must decompose the Softmax operator and implement it with the microprocessor, Host CPU and ASIC dedicated circuit to meet the customer's requirements for deploying the model first, and at the same time inform the back-end of the reasons that affect the performance bottleneck of the model. The back-end can design the target file corresponding to the accelerated Softmax operator based on the analysis and synchronize it to the eFPGA acceleration library. In this embodiment, the eFPGA program can be updated by online loading, and the Softmax operator can be subsequently implemented by the eFPGA, thereby improving system performance and user experience.

[0119] Similarly, if there is a target file for accelerating the Layer Normalization operator in the eFPGA acceleration library file, the target file for accelerating Layer Normalization can be loaded into the eFPGA, and the eFPGA can be used for acceleration when performing the LayerNormalization operation. If there is no target file for accelerating the Layer Normalization operator in the acceleration library, the compiler must decompose the Layer Normalization operator and implement it with the microprocessor, Host CPU and ASIC dedicated circuit to meet the customer's requirements for deploying the model first. At the same time, the reasons affecting the performance bottleneck of the model are informed to the rear, and the rear can design the target file for accelerating the Layer Normalization operator based on the analysis and synchronize it to the eFPGA acceleration library. In this embodiment, the eFPGA program can be updated by online loading, and the LayerNormalization operator is subsequently implemented with eFPGA, thereby improving system performance and user experience.

[0120] Name the target files in the acceleration library (it is recommended to name them after the operator) and number them to facilitate the compiler to reasonably compile and allocate tasks according to the resources of the acceleration library. For example, implement coarse-grained operators such as Softmax, LayerNorm, Sigmoid, or abstract finer-grained operators, and name and number the target files of these operators after implementation.

[0121] Furthermore, if a model has multiple operators that cannot be effectively accelerated by the engines of microprocessors and ASIC circuits, for example, the Softmax and Layer Normalization operators in the Transformer model cannot be effectively accelerated, then both operator designs can be implemented using eFPGA and unified target files can be generated and put into the acceleration library for calling, or partitioned in the eFPGA and divided into different areas to load multiple different operators.

[0122] Of course, the above descriptions are methods that can be used when the eFPGA resources are sufficient. If a model has multiple operators that need to be accelerated by the eFPGA, and if there are not enough resources to put all these operators in the eFPGA, the following methods can be used:

[0123] 1. Sort operators by their contribution to model acceleration. Operators with the greatest impact on overall performance have the highest priority, and operators with the least impact on overall performance have the lowest priority. Operators are implemented in order from high priority to low priority until the eFPGA resources are fully occupied. These high-priority operators are integrated to generate target files and loaded into the eFPGA.

[0124] 2. Analyze these operators and extract common features for acceleration. For example, multiple operators contain e x , lnx, sqrt, 1 / x, trigonometric functions, inverse trigonometric functions, etc. If these operations can be implemented, multiple operators can be accelerated.

[0125] The above description is for the basic structure of Transformer. These methods can be used to accelerate CNN, RNN and subsequent AI new technologies, new models, and new operators, and the mechanism is equally effective.

[0126] It should be noted that what is disclosed above are only several specific implementation scenarios of the acceleration method of the AI ​​processor in the embodiment of the present application. Of course, this cannot be used to limit the protection scope of the present application. Ordinary technicians in this field can understand that all or part of the processes of the above embodiments and equivalent changes made in accordance with the claims of the present invention are still within the scope covered by the present application.

[0127] That is to say, the above-mentioned specific embodiment 2 is only used to help understand the technical concept or technical principle of the embodiment of the present application, and does not constitute a limitation of the present application. More simple transformations based on the technical concept should all be within the scope of protection of the present application.

[0128] In addition, the present application also provides an acceleration system for an AI processor, referring to Figure 7 , Figure 7 This is a schematic diagram of the functional modules of the acceleration system of the AI ​​processor according to an embodiment of the present application.

[0129] In this embodiment, the acceleration system of the AI ​​processor includes:

[0130] A compiling module 10 is configured to, after identifying a target model that cannot be effectively accelerated by the AI ​​processor, determine a first operator in the target model that cannot be effectively accelerated;

[0131] The acceleration module 30 is configured to determine a target file corresponding to the first operator, and synchronize the target file to the operator acceleration library of the eFPGA of the AI ​​processor to accelerate the first operator through the eFPGA.

[0132] In some embodiments, the acceleration system of the AI ​​processor further includes:

[0133] The scheduling module 20 is configured to collaboratively execute the first operator through the microprocessor and ASIC circuit of the AI ​​processor.

[0134] In some embodiments, the compiling module 10 is further configured to:

[0135] Parsing the target model through a compiler to obtain multiple operators to be accelerated;

[0136] An operator that cannot be effectively run on the engine resources of the AI ​​processor is determined from the operators to be accelerated as a first operator that cannot be effectively accelerated in the target model.

[0137] In some embodiments, the compiling module 10 is further configured to:

[0138] The compiler maps each operator to be accelerated to the engine resources for execution based on the engine resources in the AI ​​processor in combination with a preset scheduling algorithm;

[0139] An operator to be accelerated that does not include the eFPGA engine in the mapped engine resources is used as a second operator, and it is identified whether the second operator can be effectively accelerated by the microprocessor and / or ASIC circuit of the AI ​​processor;

[0140] When it is identified that the second operator cannot be effectively accelerated, the second operator is used as an operator among the operators to be accelerated that cannot be effectively run on the engine resources of the AI ​​processor.

[0141] In some embodiments, the engine resources include the eFPGA, the microprocessor, and the engine of the ASIC circuit, and the compiling module 10 is further configured to:

[0142] Identify the engine resources in the AI ​​processor through the compiler to determine whether the engine of the ASIC circuit can accelerate each operator to be accelerated;

[0143] The operator to be accelerated that cannot be accelerated by the engine of the ASIC circuit is used as a third operator, and it is determined whether the engine of the eFPGA can accelerate the third operator;

[0144] The third operator that cannot be accelerated by the engine of the eFPGA is mapped to the engines of the microprocessor and the ASIC circuit through a preset scheduling algorithm for collaborative execution.

[0145] In some embodiments, the acceleration module 30 is further configured to:

[0146] Sending notification information to the compiler, wherein the notification information is used to trigger the compiler to identify and process the engine resources in the AI ​​processor.

[0147] In some embodiments, the acceleration module 30 is further configured to:

[0148] Outputting the information of the first operator;

[0149] Acquire an acceleration unit execution file corresponding to the first operator input based on the output prompt, wherein the acceleration unit execution file is executable by the eFPGA;

[0150] The acceleration unit execution file is used as a target file corresponding to the first operator.

[0151] In some embodiments, the acceleration module 30 is further configured to:

[0152] When it is identified that there are multiple first operators in the target model and the engine based on the eFPGA cannot execute and process each of the first operators at the same time, determine the priority of each of the first operators;

[0153] In order of priority from high to low, each of the first operators is filled into the resource space of the eFPGA engine for execution processing, and the remaining other first operators after the resource space is filled are collaboratively executed through the microprocessor of the AI ​​processor, the ASIC circuit and the eFPGA engine.

[0154] The acceleration system of the AI ​​processor provided in this embodiment and the acceleration method of the AI ​​processor provided in the above embodiment belong to the same inventive concept. For technical details not fully described in this embodiment, please refer to the embodiment of the acceleration method of the above AI processor. This embodiment has the same beneficial effects as the embodiments of the acceleration method of the AI ​​processor, which will not be repeated here.

[0155] The system embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0156] In addition, the present application also provides a chip, referring to Figure 8 , Figure 8 The hardware structure diagram of a chip provided in the embodiment of the present application is shown in FIG. Figure 8 As shown, the chip may include: a processor 1001, a communication bus 1002, a user interface 1003, a network interface 1004 and a memory 1005. Among them, the processor 1001 may be a central processing unit (CPU). The communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM), or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk storage. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0157] Those skilled in the art will understand that Figure 8 The structure shown in the figure does not constitute a limitation on the chip, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Figure 8As shown, the memory 1005 as a storage medium may include an operating system, a data storage module, a network communication module, a user interface module and an acceleration program.

[0158] exist Figure 8 In the chip shown, the network interface 1004 is mainly used for data communication with other devices; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in this embodiment can be set in the chip, and the chip calls the acceleration program stored in the memory 1005 through the processor 1001, and executes the acceleration method applied to the AI ​​processor provided in any of the above embodiments.

[0159] The device proposed in this embodiment and the acceleration method applied to the AI ​​processor proposed in the above embodiment belong to the same inventive concept. The technical details not fully described in this embodiment can be referred to any of the above embodiments, and this embodiment has the same beneficial effects as the acceleration method for executing the AI ​​processor.

[0160] In addition, an embodiment of the present application also proposes a computer-readable storage medium, which may be a non-volatile computer-readable storage medium, on which an acceleration program for an AI processor is stored, and when the acceleration program for the AI ​​processor is executed by the processor, the acceleration method for the AI ​​processor of the present application as described above is implemented.

[0161] The various embodiments of the chip and computer-readable storage medium of the present application can all refer to the various embodiments of the acceleration method of the AI ​​processor of the present application, which will not be repeated here.

[0162] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0163] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0164] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a computer-readable storage medium (such as ROM / RAM, disk, CD) as described above, and includes a number of instructions for a communication device to execute the methods described in each embodiment of the present application.

[0165] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for accelerating an AI processor, characterized in that: The method comprises: After identifying a target model that cannot be effectively accelerated by the AI ​​processor, determining a first operator in the target model that cannot be effectively accelerated; Determine a target file corresponding to the first operator, and synchronize the target file to an operator acceleration library of the eFPGA of the AI ​​processor to accelerate the first operator through the eFPGA.

2. The AI ​​processor acceleration method according to claim 1, wherein: After the step of determining the first operator in the target model that cannot be effectively accelerated, the method further includes: The first operator is collaboratively executed by the microprocessor and ASIC circuit of the AI ​​processor.

3. The AI ​​processor acceleration method according to claim 1, wherein: The step of determining the first operator in the target model that cannot be effectively accelerated comprises: Parsing the target model through a compiler to obtain multiple operators to be accelerated; An operator that cannot be effectively run on the engine resources of the AI ​​processor is determined from the operators to be accelerated as a first operator that cannot be effectively accelerated in the target model.

4. The AI ​​processor acceleration method according to claim 3, wherein: After the step of parsing the target model through a compiler to obtain a plurality of operators to be accelerated, the method further includes: The compiler maps each operator to be accelerated to the engine resources for execution based on the engine resources in the AI ​​processor in combination with a preset scheduling algorithm; An operator to be accelerated that does not include the eFPGA engine in the mapped engine resources is used as a second operator, and it is identified whether the second operator can be effectively accelerated by the microprocessor and / or ASIC circuit of the AI ​​processor; When it is identified that the second operator cannot be effectively accelerated, the second operator is used as an operator among the operators to be accelerated that cannot be effectively run on the engine resources of the AI ​​processor.

5. The AI ​​processor acceleration method according to claim 4, wherein: The engine resources include the eFPGA, the microprocessor, and the engine of the ASIC circuit. The step of mapping each operator to be accelerated to the engine resources for execution processing by the compiler based on the engine resources in the AI ​​processor in combination with a preset scheduling algorithm includes: Identify the engine resources in the AI ​​processor through the compiler to determine whether the engine of the ASIC circuit can accelerate each operator to be accelerated; The operator to be accelerated that cannot be accelerated by the engine of the ASIC circuit is used as a third operator, and it is determined whether the engine of the eFPGA can accelerate the third operator; The third operator that cannot be accelerated by the engine of the eFPGA is mapped to the engines of the microprocessor and the ASIC circuit through a preset scheduling algorithm for collaborative execution.

6. The AI ​​processor acceleration method according to claim 5, wherein: After the step of synchronizing the target file to the operator acceleration library of the eFPGA of the AI ​​processor, the method further includes: Sending notification information to the compiler, wherein the notification information is used to trigger the compiler to identify and process the engine resources in the AI ​​processor.

7. The AI ​​processor acceleration method according to claim 1, wherein: The step of determining the target file corresponding to the first operator includes: Outputting the information of the first operator; Acquire an acceleration unit execution file corresponding to the first operator input based on the output prompt, wherein the acceleration unit execution file is executable by the eFPGA; The acceleration unit execution file is used as the target file corresponding to the first operator.

8. The AI ​​processor acceleration method according to claim 1, wherein: The method further comprises: When it is identified that there are multiple first operators in the target model and the engine based on the eFPGA cannot execute and process each of the first operators at the same time, determine the priority of each of the first operators; In order of priority from high to low, each of the first operators is filled into the resource space of the eFPGA engine for execution processing, and the remaining other first operators after the resource space is filled are collaboratively executed through the microprocessor of the AI ​​processor, the ASIC circuit and the eFPGA engine.

9. The AI ​​processor acceleration method according to claim 8, wherein: The step of determining the priority of each of the first operators comprises: Identifying the influence weight of each of the first operators on the task processing performance of the network model; The priority of each of the first operators is determined according to the influence weight, wherein the greater the influence weight of the task processing performance is, the higher the priority is.

10. The AI ​​processor acceleration method according to any one of claims 1 to 9, characterized in that: The method further comprises: After identifying a network model that can be effectively accelerated by the AI ​​processor, all operators of the network model are collaboratively executed through the microprocessor, ASIC circuit and eFPGA of the AI ​​processor.

11. An acceleration system for an AI processor, characterized in that: include: A compiling module, configured to, after identifying a target model that cannot be effectively accelerated by the AI ​​processor, determine a first operator in the target model that cannot be effectively accelerated; An acceleration module is configured to determine a target file corresponding to the first operator, and synchronize the target file to an operator acceleration library of an eFPGA of the AI ​​processor to accelerate the first operator through the eFPGA.

12. A chip, characterized in that: include: A memory, a processor, and an acceleration program stored in the memory and executable on the processor, wherein the acceleration program, when executed by the processor, implements the acceleration method as described in any one of claims 1 to 10.

Images