Ai processor acceleration method and system, and chip

Abstract

The present application relates to the technical field of electronics, and discloses an AI processor acceleration method and system, and a chip. The technical solution of the AI processor acceleration method of the present application comprises: when a target model in which an AI processor cannot be effectively accelerated is identified, determining a first operator, that cannot be effectively accelerated, in the target model; and determining a target file corresponding to the first operator, and synchronizing the target file to an operator acceleration library of an eFPGA of the AI processor, so as to accelerate the first operator by means of the eFPGA.

Description

AI processor acceleration method, system and chip

[0001] Related applications

[0002] This application claims priority to Chinese patent application No. 202311182204.7 filed on September 13, 2023, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of electronic technology, and in particular to an acceleration method, system, and chip for an AI processor. Background Art

[0004] In recent years, deep learning has made significant progress in intelligent perception. In areas such as speech, image, and video processing, AI processing efficiency and effectiveness have surpassed those of humans. With the rapid development of large models over the past two years, AI has also made significant progress in language understanding and processing.

[0005] Because deep learning algorithms are data-independent and parallelizable, and their computational type is intensive, they are very suitable for hardware acceleration, which has promoted the vigorous development of AI (Artificial Intelligence) processors.

[0006] However, with the continuous emergence of new AI technologies, AI models and operators are constantly evolving, significantly impacting DSA's customized AI processors. This is manifested in an inability to effectively accelerate the execution of new models and operators, impacting the effective lifespan and competitiveness of AI processors. Furthermore, for models and operators that AI processors cannot effectively process or do not support, they require interaction with the CPU (Central Processing Unit / Processor), or even exclusive CPU execution, resulting in extremely low efficiency and failing to meet the demands of scenarios with high real-time requirements. In other words, the low adaptability and processing efficiency of AI processors for new models and operators in new AI technologies has become a pressing issue that needs to be addressed.

[0007] Summary of the Invention

[0008] The main purpose of this application is to provide an acceleration method, system and chip for an AI processor, aiming to solve the technical problems of low adaptability and processing energy efficiency of current AI processors.

[0009] To achieve the above objectives, an embodiment of the present application provides an acceleration method for an AI processor, the method comprising:

[0010] After identifying a target model that cannot be effectively accelerated by the AI ​​processor, determining a first operator in the target model that cannot be effectively accelerated;

[0011] collaboratively executing the first operator through a microprocessor and an ASIC circuit of the AI ​​processor;

[0012] Determine a target file corresponding to the first operator, and synchronize the target file to an operator acceleration library of the eFPGA of the AI ​​processor to accelerate the first operator through the eFPGA.

[0013] In addition, to achieve the above objectives, an embodiment of the present application further provides an AI processor acceleration system, including:

[0014] a compiling module configured to, after identifying a target model that cannot be effectively accelerated by the AI ​​processor, determine a first operator in the target model that cannot be effectively accelerated;

[0015] An acceleration module is configured to determine a target file corresponding to the first operator and synchronize the target file to an operator acceleration library of the eFPGA of the AI ​​processor to accelerate the first operator through the eFPGA.

[0016] In addition, an embodiment of the present application also provides a chip, including: a memory, a processor, and an acceleration program stored in the memory and runnable on the processor. When the acceleration program is executed by the processor, the acceleration method of the AI ​​processor as described above is implemented.

[0017] In addition, an embodiment of the present application also provides a computer-readable storage medium, on which an acceleration program for an AI processor is stored. When the acceleration program for the AI ​​processor is executed by a processor, the steps of the AI ​​processor acceleration method described above are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0019] FIG1 is a flow chart of a first embodiment of an acceleration method for an AI processor of the present application;

[0020] FIG2 is a flow chart of a second embodiment of an acceleration method for an AI processor of the present application;

[0021] FIG3 is a schematic structural diagram of an AI processor according to a first embodiment of the present application;

[0022] FIG4 is a schematic structural diagram of an AI processor according to a second embodiment of the present application;

[0023] FIG5 is a schematic structural diagram of an AI processor according to a third embodiment of the present application;

[0024] FIG6 is a flowchart of the AI ​​processor according to a specific embodiment of the present application;

[0025] FIG7 is a schematic diagram of the functional modules of the acceleration system of the AI ​​processor according to an embodiment of the present application;

[0026] FIG8 is a schematic diagram of the hardware structure of the chip involved in the embodiment of the present application.

[0027] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0028] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0029] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0030] It should be noted that all directional indications in the embodiments of the present application (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0031] In this application, unless otherwise specified or limited, the terms "connection" and "fixation" should be understood in a broad sense. For example, "fixation" can mean fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; internal communication between two elements or interaction between two elements, unless otherwise specified. For those skilled in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0032] In addition, the descriptions of "first", "second", etc. in this application are for descriptive purposes only and should not be understood as indicating or implying their relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but this must be based on the fact that they can be implemented by ordinary technicians in this field. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0033] Because deep learning algorithms are data-independent and parallelizable, and their computational type is intensive, they are very suitable for hardware acceleration, which has promoted the vigorous development of AI (Artificial Intelligence) processors.

[0034] However, with the continuous emergence of new AI technologies, AI models and operators are constantly evolving, significantly impacting DSA's customized AI processors. This is manifested in an inability to effectively accelerate the execution of new models and operators, impacting the effective lifespan and competitiveness of AI processors. Furthermore, for models and operators that AI processors cannot effectively process or do not support, they require interaction with the CPU (Central Processing Unit / Processor), or even exclusive CPU execution, resulting in extremely low efficiency and failing to meet the demands of scenarios with high real-time requirements. In other words, the low adaptability and processing efficiency of AI processors for new models and operators in new AI technologies has become a pressing issue that needs to be addressed.

[0035] To improve the competitiveness of AI processors, related technologies integrate multiple multi-core microprocessors within the AI ​​processor. These microprocessors are then used to implement new operators that cannot be processed. This is extremely inefficient, and the large number of integrated microprocessors leads to high AI processor costs. Some even require processing by the host CPU outside the AI ​​processor, requiring extensive interaction between the AI ​​processor and the host, resulting in low efficiency.

[0036] Based on this, an embodiment of the present application provides an acceleration method for an AI processor. Referring to FIG1 , FIG1 is a flow chart of an embodiment of an acceleration method for an AI processor of the present application. In this embodiment, the acceleration method for the AI ​​processor includes:

[0037] Step S10, after identifying a target model that cannot be effectively accelerated by the AI ​​processor, determining a first operator in the target model that cannot be effectively accelerated;

[0038] As those skilled in the art will appreciate, an AI (Artificial Intelligence) processor is an integrated circuit chip specifically designed for AI applications. It accelerates the computation and processing of AI algorithms at the hardware level, improving their efficiency and performance. AI processors typically include multiple core components, such as a processor, memory, and accelerators, enabling a variety of AI applications, including image recognition, natural language processing, speech recognition, machine translation, and autonomous driving.

[0039] In this embodiment, the target model refers to a model that the AI ​​processor cannot effectively accelerate. It is understandable that the user's model to be deployed can be automatically deployed to the AI ​​processor for acceleration after passing through the compiler. Among them, the user's model to be deployed may be a model trained based on frameworks such as Tensorflow / Caffe / Pytorch / PP Feijiang. The various processing engines of the AI ​​processor (which may include engines of eFPGA, microprocessor and ASIC circuit) execute asynchronously and in parallel under the control of the compilation instructions to complete the acceleration of the entire model, as shown in Figure 2. When the user's model to be deployed cannot be effectively accelerated by the various processing engines of the AI ​​processor, the user's model to be deployed is determined to be a target model that the AI ​​processor cannot effectively accelerate.

[0040] In one embodiment, when the AI ​​processor is accelerating the model to be deployed by the user, if the engine resource utilization rate of the AI ​​processor is less than a preset ratio threshold, the model is identified as a target model that cannot be effectively accelerated. In another embodiment, if each processing engine of the AI ​​processor fails to complete the acceleration of the model to be deployed by the user within a predetermined time period, the model is identified as a target model that cannot be effectively accelerated. In yet another embodiment, the target model that cannot be effectively accelerated may be a manually marked model, and the system can identify whether the current model is a target model that cannot be effectively accelerated by the AI ​​processor by determining whether the model carries the identification information of the manual mark. This embodiment does not specifically limit this.

[0041] Exemplarily, the step of determining a first operator in the target model that cannot be effectively accelerated includes:

[0042] Step A10: parsing the target model through a compiler to obtain multiple operators to be accelerated;

[0043] Step A20: Determine, from the operators to be accelerated, an operator that cannot be effectively run on the engine resources of the AI ​​processor, as the first operator that cannot be effectively accelerated in the target model.

[0044] In one embodiment, when the AI ​​processor is accelerating the operator to be accelerated, if the engine resource utilization rate of the AI ​​processor is less than a preset ratio threshold, it is identified that the operator to be accelerated cannot be effectively run on the engine resources of the AI ​​processor, that is, the operator to be accelerated is the first operator that cannot be effectively accelerated. In another embodiment, each processing engine of the AI ​​processor fails to complete the acceleration of the operator to be accelerated within a predetermined time period, and it is identified that the operator to be accelerated cannot be effectively run on the engine resources of the AI ​​processor, that is, the operator to be accelerated is the first operator that cannot be effectively accelerated. In yet another embodiment, the operator to be accelerated that cannot be effectively accelerated may be an operator that is manually marked, and the system can identify whether the operator to be accelerated cannot be effectively run on the engine resources of the AI ​​processor (that is, determine whether the operator to be accelerated is the first operator that the AI ​​processor cannot effectively run) by judging whether the operator to be accelerated carries the marking information of the manual marking. This embodiment does not make specific limitations on this.

[0045] AI processors can be packaged as PCIe (peripheral component interconnect express) cards and plugged into servers, allowing them to function as AI servers. Multiple PCIe cards containing AI processors can be plugged into a server to enhance its processing capabilities. AI servers can be cascaded to form AI server clusters, meeting various applications in cloud and edge scenarios.

[0046] This embodiment parses the target model through a compiler to obtain multiple operators to be accelerated, and then based on the operators in each operator that cannot be effectively run on the engine resources of the AI ​​processor, it is convenient to accurately obtain the first operator in the target model that cannot be effectively accelerated.

[0047] Step S20: Determine a target file corresponding to the first operator, and synchronize the target file to the operator acceleration library of the eFPGA of the AI ​​processor to accelerate the first operator through the eFPGA.

[0048] In this embodiment, by synchronizing the target file corresponding to the first operator to the operator acceleration library of the eFPGA of the AI ​​processor, the eFPGA can subsequently accelerate the first operator by loading the target file in the operator acceleration library.

[0049] Accelerating the first operator using the eFPGA can include four scenarios: The first scenario involves collaborative execution of the first operator using the eFPGA (Embedded FPGA) and an ASIC circuit. The second scenario involves collaborative execution of the first operator using the eFPGA and a microprocessor. The third scenario involves collaborative execution of the first operator using the eFPGA, a microprocessor, and an ASIC circuit. The fourth scenario involves executing the first operator solely using the eFPGA.

[0050] The embodiments of the present application disclose an acceleration method, system, and chip for an AI processor. The technical solution of the acceleration method for the AI ​​processor is to, after identifying a target model that the AI ​​processor cannot effectively accelerate, determine a first operator in the target model that cannot be effectively accelerated, then determine a target file corresponding to the first operator, and synchronize the target file to an operator acceleration library of an eFPGA (Embedded FPGA) of the AI ​​processor to accelerate the first operator through the eFPGA. In this way, the eFPGA is used to accelerate operators that microprocessors and ASIC circuits cannot effectively handle, effectively solving the problems of low adaptability and processing energy efficiency of the AI ​​processor in dealing with new AI technologies, new models, and new operators, thereby improving the effective market life and competitiveness of the AI ​​processor.

[0051] In a possible implementation, after step S10, determining the first operator in the target model that cannot be effectively accelerated, the method further includes:

[0052] Step B10: collaboratively executing the first operator through the microprocessor and ASIC circuit of the AI ​​processor;

[0053] Those skilled in the art will understand that an ASIC (Application Specific Integrated Circuit) circuit refers to a type of integrated circuit compared to a general-purpose integrated circuit, and is an integrated circuit designed for a specific application field or a specific customer requirement.

[0054] In this embodiment, collaborative execution of the first operator by the microprocessor and ASIC circuit of the AI ​​processor refers to: scheduling the first operator to the microprocessor and / or ASIC circuit for execution by using a preset scheduling algorithm and taking into account the engine resource space of the microprocessor and the ASIC circuit. Exemplarily, the compiler, using the preset scheduling algorithm and taking into account the engine resource space of the microprocessor and the ASIC circuit, allocates the first operator to the microprocessor for execution (when the microprocessor alone has better processing performance than the microprocessor and / or ASIC circuit, for example, when the engine resource space of the ASIC circuit is insufficient), allocates the first operator to the ASIC circuit for execution (when the ASIC circuit alone has better processing performance than the ASIC circuit and / or microprocessor, for example, when the engine resource space of the microprocessor is insufficient), or allocates the first operator to the ASIC circuit for execution (when the ASIC circuit alone has better processing performance than the ASIC circuit and / or microprocessor, for example, when the engine resource space of the microprocessor is insufficient), or allocates the first operator to the ASIC circuit and microprocessor for joint execution (when the ASIC circuit and microprocessor collaborative execution has better performance than the ASIC circuit or microprocessor alone, for example, when both the ASIC circuit and the microprocessor have a certain amount of engine resource space and the joint processing of the two can maximize the processing efficiency of the first operator).

[0055] For operators that cannot be effectively accelerated by the microprocessor and dedicated ASIC circuit of the AI ​​processor, and there is no target file corresponding to the operator in the operator acceleration library (that is, the operator is the first operator). In order not to affect the user experience, the embodiment of the present application first collaboratively executes the first operator through the microprocessor and ASIC circuit of the AI ​​processor, even if the acceleration performance is not optimal, and then determines the target file corresponding to the first operator, synchronizes the target file to the eFPGA of the AI ​​processor, and can automatically notify the form of the first operator in the rear through the compiler. The rear can use programmable logic to implement the first operator in the eFPGA and add the target file to the operator acceleration library, thereby upgrading the eFPGA program through online upgrading, so that the AI ​​processor has the best processing performance for the new operator, providing the best performance in a way that minimizes the impact on users, and further improving the adaptability and processing energy efficiency of the AI ​​processor in responding to new AI technologies, new models, and new operators.

[0056] Exemplarily, the method further includes:

[0057] In step C10, after identifying a network model that can be effectively accelerated by the AI ​​processor, all operators of the network model are collaboratively executed through the microprocessor, ASIC circuit, and eFPGA of the AI ​​processor.

[0058] In this embodiment, the collaborative execution of the first operator through the microprocessor, ASIC circuit and eFPGA of the AI ​​processor means: through a preset scheduling algorithm, taking into account the engine resource space of the microprocessor, ASIC circuit and eFPG, the first operator is scheduled to at least one of the microprocessor, ASIC circuit and eFPGA for operation.

[0059] In this embodiment, the network model refers to a model that can be effectively accelerated by the AI ​​processor. It can be understood that the model to be deployed by the user can be automatically deployed to the AI ​​processor for acceleration after passing through the compiler. Among them, the model to be deployed by the user can be a model trained based on frameworks such as Tensorflow / Caffe / Pytorch / PP Feijiang, as shown in Figure 2. The various processing engines of the AI ​​processor (which may include engines of eFPGA, microprocessor and ASIC circuit) execute asynchronously and in parallel under the control of the compilation instructions to complete the acceleration of the entire model. If the model to be deployed by the user can be effectively accelerated by the various processing engines of the AI ​​processor, it is determined that the model to be deployed by the user is a network model that can be effectively accelerated by the AI ​​processor.

[0060] After identifying a network model that can be effectively accelerated by the AI ​​processor, this embodiment collaboratively executes all operators of the network model through the AI ​​processor's microprocessor, ASIC circuit, and eFPGA. This fully utilizes all engine resources of the AI ​​processor, accelerates the deployed model, improves computing efficiency, and improves the computing power of the AI ​​processor, enabling the AI ​​processor to adapt to different scenarios and AI algorithms, increasing its flexibility, and thereby improving the adaptability and processing energy efficiency of the AI ​​processor in responding to new AI technologies, new models, and new operators.

[0061] In a possible implementation, after the step of parsing the target model using a compiler to obtain a plurality of operators to be accelerated, the method further includes:

[0062] Step D10: Mapping each operator to be accelerated to the engine resources in the AI ​​processor for execution by the compiler based on the engine resources in the AI ​​processor and in combination with a preset scheduling algorithm;

[0063] In this embodiment, the preset scheduling algorithm has been extensively studied by those skilled in the art and will not be further elaborated upon here. Furthermore, this embodiment does not impose any specific restrictions on the type of the preset scheduling algorithm; the algorithm is based on the ability to map the model operators to be deployed to the engine resources in the AI ​​processor for execution.

[0064] Step D20: The operator to be accelerated that does not include the eFPGA engine in the mapped engine resources is used as a second operator, and whether the second operator can be effectively accelerated by the microprocessor and / or ASIC circuit of the AI ​​processor is identified;

[0065] Those skilled in the art will appreciate that when executing a model operator to be deployed based on the engine resources in the AI ​​processor, the compiler determines which engine resource can effectively accelerate the operator (i.e., the model operator), for example, whether it is the eFPGA, microprocessor, or ASIC circuit, or a combination of the two. If it is determined that the eFPGA can effectively accelerate a certain model operator A, the eFPGA will prioritize the execution of the model operator A. For another example, if it is determined that the eFPGA cannot effectively accelerate a certain model operator B, the eFPGA engine will generally not prioritize the acceleration of the model operator B. In other words, the engine resources mapped to model operator B often do not include the eFPGA engine. If the engine resources mapped to model operator B do not include the eFPGA engine, then model operator B is referred to as the "second operator" above, and the compiler will then determine whether the microprocessor and / or ASIC circuit can effectively accelerate the second operator.

[0066] Step D30: When it is identified that the second operator cannot be effectively accelerated, the second operator is used as an operator among the operators to be accelerated that cannot be effectively run on the engine resources of the AI ​​processor.

[0067] In this embodiment, if it is identified that the second operator cannot be effectively accelerated by the microprocessor and / or ASIC circuit, it means that the second operator cannot be effectively accelerated by all engine resources of the AI ​​processor. At this time, the second operator is the operator among all operators that cannot be effectively run on the engine resources of the AI ​​processor.

[0068] In this embodiment, the compiler uses the engine resources in the AI ​​processor and a preset scheduling algorithm to map each operator to the engine resources for execution. The operator to be accelerated that does not include the eFPGA engine in the mapped engine resources is treated as the second operator. Then, it is identified whether the second operator can be effectively accelerated by the microprocessor and / or ASIC circuit. If it is identified that the second operator cannot be effectively accelerated, the second operator is used as the operator that cannot be effectively run on the engine resources of the AI ​​processor, thereby facilitating further and more accurate identification of the first operator in the target model that cannot be effectively accelerated.

[0069] In one possible implementation, the engine resources include the eFPGA, the microprocessor, and the engine of the ASIC circuit. The step of mapping each operator to be accelerated to the engine resources for execution by the compiler based on the engine resources in the AI ​​processor in combination with a preset scheduling algorithm includes:

[0070] Step E10: Identifying engine resources in the AI ​​processor through a compiler to determine whether the engine of the ASIC circuit is capable of accelerating each operator to be accelerated;

[0071] Step E20: Using the operator to be accelerated that cannot be accelerated by the engine of the ASIC circuit as a third operator, and determining whether the engine of the eFPGA can accelerate the third operator;

[0072] Step E30: Map the third operator that cannot be accelerated by the eFPGA engine to the engines of the microprocessor and the ASIC circuit through a preset scheduling algorithm for collaborative execution.

[0073] The engine resources in the AI ​​processor include the eFPGA engine (the eFPGA engine is the "eFPGA engine" mentioned above), the microprocessor engine, and the ASIC circuit engine.

[0074] In this embodiment, if the operator to be accelerated can be effectively accelerated by the ASIC circuit, the ASIC circuit is used for effective acceleration first. The remaining operators need to determine whether the eFPGA engine resources (operator acceleration library) can be accelerated. The operators that cannot be accelerated by the eFPGA engine among the remaining operators are the second operators.

[0075] In this embodiment, if an operator C needs to be accelerated by the eFPGA engine, the eFPGA operator acceleration library must store the target file corresponding to operator C. For ease of description, the target file corresponding to operator C is referred to as target file c. The eFPGA engine can then accelerate operator C by loading target file c from the operator acceleration library. In addition, a special case needs to be considered. Even if the target file c exists in the operator acceleration library, if the resource space of the eFPGA engine is fully occupied, it is impossible to free up sufficient engine resources (that is, the computing power resources of the eFPGA) to load the target file c into the operator acceleration library to accelerate operator C. At this time, effective acceleration of operator C cannot be achieved. Therefore, this embodiment uses the compiler to identify the eFPGA operator acceleration library (that is, to identify whether there is a corresponding target file for the eFPGA operator acceleration library). It also combines the eFPGA engine to identify whether the eFPGA engine can accelerate each operator (that is, it also comprehensively considers whether the computing power resources of the eFPGA engine are sufficient), thereby improving the accuracy of identifying whether the current model operator can be accelerated by the eFPGA engine.

[0076] In one possible implementation, after the step of synchronizing the target file to the operator acceleration library of the eFPGA of the AI ​​processor, the method further includes:

[0077] Step F10: Send notification information to the compiler, wherein the notification information is used to trigger the compiler to identify and process the engine resources in the AI ​​processor.

[0078] In this embodiment, after the target file is added to the operator acceleration library (that is, after the eFPGA program is upgraded), the eFPGA engine now has the ability to execute the first operator (the target file added to the operator acceleration library is the target file corresponding to the first operator), or in other words, the eFPGA has the ability to accelerate the first operator. At this time, after the step of synchronizing the target file to the eFPGA operator acceleration library of the AI ​​processor, this embodiment also sends a notification message to the compiler, and the compiler responds to the notification message and executes again: the step of identifying and processing the eFPGA operator acceleration library by the compiler, so that the compiler can re-identify and recognize the target file of the first operator in the eFPGA operator acceleration library, that is, recognize that the eFPGA has The AI ​​processor has the ability to accelerate the first operator. At this time, the original operator acceleration logic of "collaborative execution of the first operator through the AI ​​processor's microprocessor and ASIC circuit" is converted to "collaborative execution through the microprocessor, ASIC circuit and eFPGA engine". Since the eFPGA engine is added for acceleration, this embodiment fully utilizes all engine resources of the AI ​​processor to accelerate the deployed model, improve computing efficiency, and improve the computing power of the AI ​​processor. This embodiment upgrades the eFPGA program through online upgrades to optimize the AI ​​processor's processing performance for the new operator, providing optimal performance with minimal impact on users, thereby improving the AI ​​processor's adaptability and processing energy efficiency to new AI technologies, new models, and new operators.

[0079] This embodiment notifies the compiler to re-evaluate the processing engine resources of the AI ​​processor after synchronizing the circuit acceleration library of the eFPGA. The eFPGA reloads the appropriate circuit through online download, and the compiler redistributes the tasks handled by each processing engine to achieve optimized performance and improve the robustness of the AI ​​processor.

[0080] In one practicable manner, the step of determining the target file corresponding to the first operator includes:

[0081] Step G10, outputting the information of the first operator;

[0082] Step G20: obtaining an acceleration unit execution file corresponding to the first operator input based on the output prompt, wherein the acceleration unit execution file is executable by the eFPGA;

[0083] Step G30: Use the acceleration unit execution file as the target file corresponding to the first operator.

[0084] This embodiment outputs the information of the first operator, thereby automatically notifying the back-end (i.e., the back-end engineer) of the form of the first operator, so that the back-end can implement the first operator in the eFPGA using programmable logic, and then obtain the acceleration unit execution file corresponding to the first operator input based on the output prompt (the acceleration unit execution file is produced by the back-end based on the form of the first operator), wherein the acceleration unit execution file is executable by the eFPGA, and then the acceleration unit execution file is used as the target file corresponding to the first operator, thereby facilitating the addition of the target file to the operator acceleration library, effectively upgrading the eFPGA program based on online upgrade, and ensuring the optimal processing performance of the AI ​​processor for the new operator.

[0085] In order to help understand the technical concept or technical principle of the embodiments of the present application, a specific embodiment 1 is listed:

[0086] This specific embodiment provides an AI processor that includes a microprocessor (ARM, RISC-V, or self-developed), a dedicated ASIC circuit, and an eFPGA. The microprocessor, ASIC circuit, and eFPGA are organically connected via an internal bus such as a Nominal Component Interconnect (NOC), AXI4, or AHB. The eFPGA is used to accelerate operators that the microprocessor and ASIC circuit cannot effectively handle. This effectively addresses the challenges of adaptability and processing energy efficiency for new AI technologies, models, and operators, thereby enhancing the market lifespan and competitiveness of the AI ​​processor.

[0087] Figure 3 shows the connection between the ASIC circuit and the eFPGA. The control and management channel connecting the microprocessor in the ASIC circuit to the eFPGA is responsible for loading and updating the eFPGA target program and for reset control management. The dedicated ASIC circuit and the eFPGA have a synchronization control channel and a data exchange channel. The synchronization control channel is responsible for transmitting synchronous information for collaborative processing between the microprocessor, dedicated ASIC circuit, and eFPGA, while the data exchange channel is responsible for transmitting collaborative data between the microprocessor, dedicated ASIC circuit, and eFPGA.

[0088] Figure 4 shows the connection between the dedicated ASIC and the eFPGA. The dedicated ASIC contains efficient processing units tailored for AI computing, such as a matrix computation engine, a vector processing engine, and a data movement and transformation engine. These engines exchange data through buffers, driving computation through data flows. The dedicated ASIC and the eFPGA have a synchronization control channel and a data exchange channel. The synchronization control channel is responsible for synchronizing information transmission for collaborative processing between the microprocessor, dedicated ASIC, and eFPGA, while the data exchange channel is responsible for transmitting data for collaborative processing between the microprocessor, dedicated ASIC, and eFPGA.

[0089] Among them, the microprocessor can be composed of ARM, RISC-V or self-developed microarchitecture. The AISC circuit includes a matrix (tensor) processing engine (Tensor Engine), a vector processing engine (Vector Engine) and a data transfer and transformation engine (Data Switch Engine). The microprocessor is used for instruction distribution and control, the Tensor Engine is used to process matrix (tensor) operations, and the Vector Engine is used to process vector operations. The main function of the data transfer and transformation engine is to transform data and realize efficient data transfer between multiple buffers and between buffers and DDR / HBM.

[0090] The matrix (tensor) computing engine, vector computing engine, data movement and transformation engine, and eFPGA processing unit need to receive instructions distributed by the microprocessor engine to complete various predefined instruction operations. The microprocessor provides target file update channels and control channels for the eFPGA, can reset the eFPGA or ASIC circuit, and can also process special complex operations that the ASIC circuit and eFPGA cannot handle.

[0091] The ASIC circuitry features multiple parallel acceleration engines: a matrix (tensor) computation engine, a vector computation engine, and a data movement and transformation engine. The matrix (tensor) computation engine extracts activations and weights from the Fractal Buffer and Weight Buffer, respectively, to perform matrix multiplications, storing the results in the Middle Buffer for partial summation or output. The vector computation engine is responsible for processing operations other than convolution, including max pooling, average pooling, relu (Real Unit) operations, prelude (Prelude) operations, normalization, scaling, and quantization.

[0092] eFPGAs are used to accelerate operators that microprocessors and ASICs cannot effectively handle. The eFPGA circuit acceleration targets operator units in models such as CNNs, RNNs, and Transformers that cannot be effectively processed by microprocessors and ASICs. These operators are categorized and abstracted, implemented using the eFPGA, and multiple target files are generated and stored in the operator acceleration library. Based on the actual deployment model, the target files for the corresponding operators are loaded into the eFPGA for acceleration.

[0093] The eFPGA acceleration library file can be stored on a server and transferred to the AI ​​processor's DDR (Double Data Rate) / HBM (High Bandwidth Memory) memory via the PCIe interface (i.e., bus interface). The microprocessor reads the target file from the DDR / HBM and loads it into the eFPGA via the GPIO (General-Purpose Input / Output Ports) interface, simulating the JTAG (Joint Test Action Group) interface timing. If needed, the microprocessor can reset the eFPGA through the reset control channel. The eFPGA has an Operator Cmd Queue to store instructions for acceleration operators implemented in the eFPGA, and a Data Buffer to store data to be processed by the Operator_1Function...Operator_nFunction operator acceleration units, as well as the results returned after processing. Operators (Operator_1Function...Operator_nFunction) can be cascaded to implement more complex operators. Operator_1Function...Operator_nFunction can be updated online to other functions based on the current model's acceleration requirements. Data buffers can be configured as multiple ping-pong buffers to reduce buffer access conflicts.

[0094] The principle block diagram of the AI ​​processor is shown in Figure 5. The microprocessor, eFPGA, and different processing engines exchange data through buffers, driving calculations through data flow and improving operational efficiency.

[0095] In this embodiment, the microprocessor controls the loading and resetting of the eFPGA. The eFPGA acceleration circuit and the various acceleration engines (matrix / tensor processing engines, vector processors, and scalar processors) in the ASIC circuit exchange data and work together via a ping-pong buffer. The compiler breaks the model into micro-units that can be processed by these processing engines, and the microprocessor sends them to each processing engine for coordinated and efficient processing.

[0096] Given the rapidly changing nature of new AI technologies, new models, and new operators, new operators that cannot be effectively accelerated can be effectively addressed by continuously designing and implementing new operator circuits, generating target files, and synchronizing them to the acceleration library. This can effectively address the low processing efficiency of DSA (Domain Specific Architecture) customized AI chips caused by the endless emergence of new technologies, new models, and new operators.

[0097] In this embodiment, for operators that cannot be effectively processed by microprocessors and ASIC circuits, eFPGA (embedded programmable logic) is used to create acceleration modules, generate target files, and integrate these operator acceleration modules into an operator acceleration library.

[0098] For models processed by the AI ​​processor, the compiler first parses the model to identify the operators that need to be accelerated. It also determines whether the current AI processor resources can effectively process the model. Processor resources include the microprocessor, ASIC circuit, and the operator acceleration library corresponding to the eFPGA. If the model can be effectively processed, the target file of the corresponding operator module is loaded into the eFPGA. The microprocessor, ASIC circuit, and eFPGA effectively cooperate to perform efficient training or inference tasks of the model. Otherwise, the compiler points out that the current resources cannot effectively process the operator. The corresponding operator can be added through design and implemented using eFPGA resources. After the design is completed, the target file is generated and synchronized to the operator acceleration library. In the subsequent processing involving the operator, the operator target file can be directly loaded for effective acceleration.

[0099] For operators that cannot be effectively accelerated in future AI technologies, models, and operators, eFPGA resources can be used for acceleration, and the target files after implementation can be stored in the operator acceleration library. Because eFPGA is programmable, these operators implemented with eFPGA can be subsequently iterated and optimized and upgraded.

[0100] For devices currently in use in the field, operators parsed by the compiler from the user model cannot be effectively accelerated by microprocessors and dedicated ASIC circuits, and there are no corresponding operator acceleration target files in the acceleration library. To avoid impacting users, the compiler first considers accelerating the operator using the engine in the microprocessor and dedicated ASIC circuit (even if the performance is not optimal). The compiler then notifies the backend of the operator form that cannot be effectively accelerated. The backend can then implement the operator in the eFPGA using programmable logic, add the target file to the acceleration library, and upgrade the eFPGA program through online upgrades to optimize the processing performance of the operator, providing optimal performance with minimal impact to users.

[0101] This specific embodiment addresses the adaptability and processing energy efficiency issues of DSA-customized AI chips in response to new AI technologies, new models, and new operators. It proposes an AI processor that internally incorporates a microprocessor (ARM, RISC-V, or self-developed), dedicated ASIC circuitry, and an eFPGA. The microprocessor, ASIC circuitry, and eFPGA are organically connected via internal buses such as the National Open Circuit (NOC), AXI4, and AHB. The eFPGA is used to accelerate operators that the microprocessor and ASIC circuitry cannot effectively handle. By classifying and abstracting new AI technologies, new models, and new operators, implementing these new operators with circuits and loading them into the eFPGA, this effectively addresses the adaptability and processing energy efficiency issues of DSA-customized AI chips in response to new AI technologies, new models, and new operators, thereby improving the market lifespan and competitiveness of AI processors.

[0102] Although the above-mentioned specific embodiments show many details, they are only used to help understand the technical concepts or technical principles of the embodiments of this application and do not constitute a limitation of this application. More simple transformations based on the technical concepts should all be within the scope of protection of this application.

[0103] In one possible implementation, the method further includes:

[0104] Step H10: determining a priority of each first operator when it is recognized that there are multiple first operators in the target model and the engine based on the eFPGA cannot execute all the first operators simultaneously;

[0105] In step H20, each of the first operators is filled into the resource space of the eFPGA engine for execution processing in descending order of priority of each of the first operators, and the remaining first operators after the resource space is filled are collaboratively executed by the microprocessor, ASIC circuit and eFPGA engine of the AI ​​processor.

[0106] Exemplarily, the step of determining the priority of each first operator includes:

[0107] Step I10: identifying the influence weight of each of the first operators on the task processing performance of the network model;

[0108] Step I20: determining the priority of each first operator according to the impact weight, wherein the greater the impact weight of the task processing performance, the higher the priority.

[0109] In this embodiment, if the compiler parses multiple operators in the AI ​​processor model that cannot be effectively processed, the operators that cannot be effectively processed are sorted by priority, with the operator with the greatest impact on overall performance being given the highest priority, and the operator with the least impact on overall performance being given the lowest priority. If multiple acceleration library operators are to be used, depending on the eFPGA's resource availability, if resources are sufficient, the design containing these operators is integrated to generate a new target file and loaded into the eFPGA. If resources are insufficient, the operators are implemented in descending order of priority, and a number of high-priority operators are integrated to generate a target file and loaded into the eFPGA.

[0110] This embodiment, upon identifying multiple first operators in a target model and the eFPGA-based engine being unable to execute all of them simultaneously, determines the priority of each first operator. Then, in descending order of priority, each first operator is placed into the eFPGA engine's resource space for execution. After the resource space is filled, the remaining first operators are collaboratively executed by the microprocessor, ASIC circuit, and eFPGA engine. This embodiment not only identifies whether the eFPGA has the ability to accelerate each operator (i.e., identifies whether a corresponding target file exists in the eFPGA's operator acceleration library), but also comprehensively considers the eFPGA engine's computing power and resource space. Furthermore, given the limited eFPGA computing power, the eFPGA determines the execution priority of each first operator based on the weight of each first operator's impact on the network model's task processing performance. This optimizes the overall processing performance of the AI ​​processor for the user-deployed network model, enabling the AI ​​processor to adapt to different scenarios and AI algorithms, improving its flexibility and further enhancing its adaptability and processing efficiency for new AI technologies, new models, and new operators.

[0111] In order to further help understand the technical concept or technical principle of the embodiments of the present application, a specific embodiment 2 is listed. The AI ​​processor workflow diagram of this specific embodiment is shown in Figure 6:

[0112] 1. For the model that the user needs to deploy, the software compiler parses the model and integrates and optimizes it, including operator fusion and parallel optimization.

[0113] 2. After parsing the model, the compiler splits the model into micro-units that can be processed by these processing engines based on the current engine resources of the AI ​​processor, including the microprocessor, the matrix (tensor) processing engine in the ASIC circuit, the vector processing engine and the data movement and transformation engine, and the circuit acceleration library processing unit of the eFPGA.

[0114] 3. If, after parsing the model, new operators are found that cannot be effectively processed by the current processing engine resources, in order not to affect user deployment, the compiler should first use one or more collaborative processing engines such as microprocessors, ASIC circuits, and acceleration units in the eFPGA circuit acceleration library to process the new operators that cannot be effectively processed. This will first meet user deployment requirements and then optimize performance through online upgrades.

[0115] 4. By printing information, the new operator that cannot be effectively processed is transmitted to the back-end. The developer analyzes the new operator that cannot be effectively processed, implements the operator using eFPGA, and adds the target file of the implemented operator to the circuit acceleration library of eFPGA.

[0116] 5. After the eFPGA circuit acceleration library is synchronized, the compiler is notified to re-evaluate the processing engine resources of the AI ​​processor. The eFPGA reloads the appropriate circuit through online download, and the compiler re-divides and distributes the tasks handled by each processing engine to achieve the goal of optimizing performance and improving the competitiveness of the AI ​​processor.

[0117] AI processors are used to accelerate the performance of deep learning models, provide powerful computing resources, speed up deployment, and improve user experience. Current deep learning models mainly include CNN, RNN, and Transformer models. This example uses the Transformer model, the basic model of chatGPT, as an example to introduce it in detail:

[0118] First, the software compiler parses the basic Transformer model and finds that deploying this model requires efficient implementation of operators such as MatMul, Scale, Softmax, Layer Normalization, and Concat.

[0119] Matmul is a matrix multiplication operation that can be efficiently implemented using the matrix (tensor) computing engine in the ASIC circuit. Scale operations can be efficiently implemented using the vector computing engine in the ASIC circuit. Concat can be efficiently implemented using the data movement and transformation engine in the ASIC circuit.

[0120] In CNN models, Softmax is typically performed in the last layer, and the amount of data required for calculation is relatively small. Therefore, using a microprocessor or sending the data to the host CPU via the PCIe interface does not present a performance bottleneck. However, in Transformer models, the Softmax operator is required in every layer, and the amount of data required for Softmax calculation is very large. Sending the data to the host CPU via the microprocessor or PCIe interface for further processing would severely impact the AI ​​processor's performance, thereby affecting its competitiveness. If the eFPGA acceleration library contains a target file for Softmax acceleration, the target file can be loaded into the eFPGA and used for Softmax acceleration. If the acceleration library does not yet contain a target file for Softmax acceleration, the compiler will decompose the Softmax operator and implement it using a combination of the microprocessor, host CPU, and ASIC circuitry. This will first meet the customer's model deployment requirements and then inform the backend of the causes of the model's performance bottlenecks. The backend can then design the target file for Softmax acceleration based on the analysis and synchronize it with the eFPGA acceleration library. In this embodiment, the eFPGA program can be updated by online loading, and the Softmax operator can be subsequently implemented using the eFPGA, thereby improving system performance and enhancing user experience.

[0121] Similarly, if the eFPGA acceleration library contains a target file for accelerating the Layer Normalization operator, the target file for accelerating Layer Normalization can be loaded into the eFPGA, and the eFPGA can be used for acceleration when performing the Layer Normalization operation. If the acceleration library does not yet contain a target file for accelerating the Layer Normalization operator, the compiler will decompose the Layer Normalization operator and implement it using a microprocessor, host CPU, and ASIC dedicated circuit to meet the customer's model deployment requirements. At the same time, the reasons affecting the model performance bottleneck are reported to the backend, who can design a target file for accelerating the Layer Normalization operator based on the analysis and synchronize it to the eFPGA acceleration library. This embodiment allows the eFPGA program to be updated through online loading, and the Layer Normalization operator can be subsequently implemented using the eFPGA, thereby improving system performance and user experience.

[0122] Name the target files in the acceleration library (it's recommended to name them after the operators) and number them to facilitate the compiler's subsequent compilation and task allocation based on the resources in the acceleration library. For example, when implementing coarse-grained operators such as Softmax, LayerNorm, and Sigmoid, or abstracted finer-grained operators, name and number the target files of these operators after implementation.

[0123] If a model contains multiple operators that cannot be effectively accelerated by the engines of microprocessors and ASIC circuits, for example, the Softmax and Layer Normalization operators in the Transformer model cannot be effectively accelerated, both operators can be designed using eFPGA and unified target files can be generated. These files can then be placed in the acceleration library for calling, or partitioned in the eFPGA to load multiple different operators into different areas.

[0124] Of course, the above descriptions are all methods that can be used when eFPGA resources are sufficient. If a model has multiple operators that require eFPGA acceleration, and if there are insufficient resources to place all these operators in the eFPGA, the following methods can be used:

[0125] 1. Operators are ranked by their contribution to model acceleration, with the operator with the greatest impact on overall performance being given the highest priority, and the operator with the least impact on overall performance being given the lowest priority. Operators are implemented in descending order until the eFPGA resources are fully utilized. These high-priority operators are then integrated to generate a target file and loaded into the eFPGA.

[0126] 2. Analyze these operators and extract common features for acceleration. For example, multiple operators contain e x , lnx, sqrt, 1 / x, trigonometric functions, inverse trigonometric functions, etc. If these operations can be implemented, multiple operators can be accelerated.

[0127] The above description applies to the basic structure of Transformer. These methods can also be used to accelerate CNN, RNN, and subsequent AI technologies, new models, and new operators. The mechanism is also effective.

[0128] The above disclosures are merely a few specific implementation scenarios for the acceleration method of the AI ​​processor in the embodiment of the present application. This is certainly not intended to limit the scope of protection of the present application. A person skilled in the art will understand that any equivalent changes made to the implementation of all or part of the processes of the above embodiment in accordance with the claims of the present application are still within the scope of the present application.

[0129] That is to say, the above-mentioned specific embodiment 2 is only used to help understand the technical concept or technical principle of the embodiment of this application, and does not constitute a limitation of this application. More simple transformations based on this technical concept should all be within the scope of protection of this application.

[0130] In addition, an embodiment of the present application also proposes an acceleration system for an AI processor. Referring to FIG. 7 , FIG. 7 is a schematic diagram of the functional modules of the acceleration system for an AI processor in an embodiment of the present application.

[0131] In this embodiment, the acceleration system of the AI ​​processor includes:

[0132] The compiling module 10 is configured to, after identifying a target model that cannot be effectively accelerated by the AI ​​processor, determine a first operator in the target model that cannot be effectively accelerated;

[0133] The acceleration module 30 is configured to determine a target file corresponding to the first operator and synchronize the target file to the operator acceleration library of the eFPGA of the AI ​​processor to accelerate the first operator through the eFPGA.

[0134] In some embodiments, the acceleration system of the AI ​​processor further includes:

[0135] The scheduling module 20 is configured to collaboratively execute the first operator through the microprocessor and ASIC circuit of the AI ​​processor.

[0136] In some embodiments, the compiling module 10 is further configured to:

[0137] Parsing the target model through a compiler to obtain multiple operators to be accelerated;

[0138] An operator that cannot be effectively run on the engine resources of the AI ​​processor is determined from the operators to be accelerated, as a first operator that cannot be effectively accelerated in the target model.

[0139] In some embodiments, the compiling module 10 is further configured to:

[0140] The compiler maps each operator to be accelerated to the engine resources in the AI ​​processor for execution based on the engine resources in combination with a preset scheduling algorithm;

[0141] An operator to be accelerated that does not include the eFPGA engine in the mapped engine resources is used as a second operator, and whether the second operator can be effectively accelerated by the microprocessor and / or ASIC circuit of the AI ​​processor is identified;

[0142] When it is identified that the second operator cannot be effectively accelerated, the second operator is used as an operator among the operators to be accelerated that cannot be effectively run on the engine resources of the AI ​​processor.

[0143] In some embodiments, the engine resources include the eFPGA, the microprocessor, and the engine of the ASIC circuit, and the compilation module 10 is further configured to:

[0144] Identifying engine resources in the AI ​​processor through a compiler to determine whether the engine of the ASIC circuit is capable of accelerating each operator to be accelerated;

[0145] An operator to be accelerated that cannot be accelerated by the engine of the ASIC circuit is used as a third operator, and determining whether the engine of the eFPGA can accelerate the third operator;

[0146] The third operator that cannot be accelerated by the engine of the eFPGA is mapped to the engines of the microprocessor and the ASIC circuit through a preset scheduling algorithm for collaborative execution.

[0147] In some embodiments, the acceleration module 30 is further configured to:

[0148] Sending notification information to the compiler, wherein the notification information is used to trigger the compiler to identify and process the engine resources in the AI ​​processor.

[0149] In some embodiments, the acceleration module 30 is further configured to:

[0150] Outputting the information of the first operator;

[0151] Obtaining an acceleration unit execution file corresponding to the first operator input based on the output prompt, wherein the acceleration unit execution file is executable by the eFPGA;

[0152] The acceleration unit execution file is used as the target file corresponding to the first operator.

[0153] In some embodiments, the acceleration module 30 is further configured to:

[0154] When it is recognized that there are multiple first operators in the target model and the engine based on the eFPGA cannot execute and process each of the first operators at the same time, determine a priority of each of the first operators;

[0155] In descending order of priority, each of the first operators is filled into the resource space of the eFPGA engine for execution and processing, and the remaining first operators after the resource space is filled are collaboratively executed by the AI ​​processor's microprocessor, ASIC circuit, and eFPGA engine.

[0156] The AI ​​processor acceleration system provided in this embodiment and the AI ​​processor acceleration method provided in the above embodiments belong to the same application concept. For technical details not fully described in this embodiment, please refer to the embodiments of the above-mentioned AI processor acceleration method. This embodiment has the same beneficial effects as the embodiments of the AI ​​processor acceleration method, and will not be repeated here.

[0157] The system embodiment described above is merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0158] In addition, an embodiment of the present application further provides a chip, with reference to FIG8 , which is a schematic diagram of the hardware structure of a chip provided in an embodiment of the present application. As shown in FIG8 , the chip may include: a processor 1001, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the processor 1001 may be a central processing unit (CPU). The communication bus 1002 is used to implement connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk storage. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0159] Those skilled in the art will appreciate that the structure shown in FIG8 does not limit the chip and may include more or fewer components than shown, or combine certain components, or arrange the components differently. As shown in FIG8 , the memory 1005 as a storage medium may include an operating system, a data storage module, a network communication module, a user interface module, and an acceleration program.

[0160] In the chip shown in Figure 8, the network interface 1004 is mainly used for data communication with other devices; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in this embodiment can be set in the chip, and the chip calls the acceleration program stored in the memory 1005 through the processor 1001, and executes the acceleration method applied to the AI ​​processor provided in any of the above embodiments.

[0161] The device proposed in this embodiment and the acceleration method applied to the AI ​​processor proposed in the above embodiment belong to the same application concept. Technical details not fully described in this embodiment can be referred to any of the above embodiments, and this embodiment has the same beneficial effects as the acceleration method for executing the AI ​​processor.

[0162] In addition, an embodiment of the present application also proposes a computer-readable storage medium, which may be a non-volatile computer-readable storage medium. The computer-readable storage medium stores an acceleration program for an AI processor. When the acceleration program for the AI ​​processor is executed by the processor, the acceleration method of the AI ​​processor of the present application as described above is implemented.

[0163] The various embodiments of the chip and computer-readable storage medium of the present application can refer to the various embodiments of the acceleration method of the AI ​​processor of the present application, and will not be repeated here.

[0164] As used herein, the terms "comprises," "comprising," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system that includes a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or system that includes the element.

[0165] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0166] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course, by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a computer-readable storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a communication device to execute the methods described in each embodiment of the present application.

[0167] The above are only some embodiments of the present application and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of this application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for accelerating an AI processor, wherein: The method comprises: After identifying a target model that cannot be effectively accelerated by the AI ​​processor, determining a first operator in the target model that cannot be effectively accelerated; Determine a target file corresponding to the first operator, and synchronize the target file to an operator acceleration library of the eFPGA of the AI ​​processor to accelerate the first operator through the eFPGA.

2. The AI ​​processor acceleration method according to claim 1, wherein: After the step of determining the first operator in the target model that cannot be effectively accelerated, the method further includes: The first operator is collaboratively executed by the microprocessor and ASIC circuit of the AI ​​processor.

3. The AI ​​processor acceleration method according to claim 1, wherein: The step of determining the first operator in the target model that cannot be effectively accelerated comprises: Parsing the target model through a compiler to obtain multiple operators to be accelerated; An operator that cannot be effectively run on the engine resources of the AI ​​processor is determined from the operators to be accelerated as a first operator that cannot be effectively accelerated in the target model.

4. The AI ​​processor acceleration method according to claim 3, wherein: After the step of parsing the target model through a compiler to obtain a plurality of operators to be accelerated, the method further includes: The compiler maps each operator to be accelerated to the engine resources for execution based on the engine resources in the AI ​​processor in combination with a preset scheduling algorithm; An operator to be accelerated that does not include the eFPGA engine in the mapped engine resources is used as a second operator, and it is identified whether the second operator can be effectively accelerated by the microprocessor and / or ASIC circuit of the AI ​​processor; When it is identified that the second operator cannot be effectively accelerated, the second operator is used as an operator among the operators to be accelerated that cannot be effectively run on the engine resources of the AI ​​processor.

5. The AI ​​processor acceleration method according to claim 4, wherein: The engine resources include the eFPGA, the microprocessor, and the engine of the ASIC circuit. The step of mapping each operator to be accelerated to the engine resources for execution processing by the compiler based on the engine resources in the AI ​​processor in combination with a preset scheduling algorithm includes: Identify the engine resources in the AI ​​processor through the compiler to determine whether the engine of the ASIC circuit can accelerate each operator to be accelerated; The operator to be accelerated that cannot be accelerated by the engine of the ASIC circuit is used as a third operator, and it is determined whether the engine of the eFPGA can accelerate the third operator; The third operator that cannot be accelerated by the engine of the eFPGA is mapped to the engines of the microprocessor and the ASIC circuit through a preset scheduling algorithm for collaborative execution.

6. The AI ​​processor acceleration method according to claim 5, wherein: After the step of synchronizing the target file to the operator acceleration library of the eFPGA of the AI ​​processor, the method further includes: Sending notification information to the compiler, wherein the notification information is used to trigger the compiler to identify and process the engine resources in the AI ​​processor.

7. The AI ​​processor acceleration method according to claim 1, wherein: The step of determining the target file corresponding to the first operator includes: Outputting the information of the first operator; Acquire an acceleration unit execution file corresponding to the first operator input based on the output prompt, wherein the acceleration unit execution file is executable by the eFPGA; The acceleration unit execution file is used as the target file corresponding to the first operator.

8. The AI ​​processor acceleration method according to claim 1, wherein: The method further comprises: When it is identified that there are multiple first operators in the target model and the engine based on the eFPGA cannot execute and process each of the first operators at the same time, determine the priority of each of the first operators; In order of priority from high to low, each of the first operators is filled into the resource space of the eFPGA engine for execution processing, and the remaining other first operators after the resource space is filled are collaboratively executed through the microprocessor of the AI ​​processor, the ASIC circuit and the eFPGA engine.

9. The AI ​​processor acceleration method according to claim 8, wherein: The step of determining the priority of each of the first operators comprises: Identifying the influence weight of each of the first operators on the task processing performance of the network model; The priority of each of the first operators is determined according to the impact weight, wherein the greater the impact weight of the task processing performance, the higher the priority The higher the level.

10. The AI ​​processor acceleration method according to any one of claims 1 to 9, wherein: The method further comprises: After identifying a network model that can be effectively accelerated by the AI ​​processor, all operators of the network model are collaboratively executed through the microprocessor, ASIC circuit and eFPGA of the AI ​​processor.

11. An acceleration system for an AI processor, wherein: The system comprises: A compiling module, configured to, after identifying a target model that cannot be effectively accelerated by the AI ​​processor, determine a first operator in the target model that cannot be effectively accelerated; An acceleration module is configured to determine a target file corresponding to the first operator, and synchronize the target file to an operator acceleration library of the eFPGA of the AI ​​processor to accelerate the first operator through the eFPGA.

12. A chip, wherein: The chip includes: a memory, a processor, and an acceleration program stored in the memory and executable on the processor, wherein the acceleration program implements the acceleration method as described in any one of claims 1 to 10 when executed by the processor.

Images