A TensorFlow framework core computing acceleration engine for domestic heterogeneous many-core processors

Abstract

The present invention discloses a TensorFlow framework core computing acceleration engine for domestic heterogeneous many-core processors. The engine is independently arranged between the TensorFlow framework and multiple many-core computing acceleration libraries, and includes the following functional modules: an operator interface module for transferring the running process to the core computing acceleration engine; an analysis and scheduling module for analyzing the operator type and operator parameter characteristics, encapsulating the operator parameters into the parameter structure of each many-core computing acceleration library, and calling different many-core computing acceleration libraries according to the operator type. When there are multiple operator implementations of the same operator type, the optimal implementation method is automatically selected and called according to the operator parameter characteristics; a debugging module for recording the running time of each operator and analyzing the performance of the model reasoning or training process from the operator level. The present invention can give full play to the powerful computing power of domestic heterogeneous many-core processors and provide automated core acceleration capabilities for deep learning tasks.

Description

Technical Field

[0001] The present invention relates to a TensorFlow framework core computing acceleration engine for domestic heterogeneous many-core processors, belonging to the field of high-performance computing technology. Background Art

[0002] Domestic heterogeneous many-core processors have super-high computing performance, and their software systems have a variety of deeply optimized computing acceleration libraries, mainly including SWDNN, SWBlas, many-core mathematics library, etc. These computing acceleration libraries can fully utilize the high performance of domestic heterogeneous many-core processors.

[0003] The TensorFlow framework is a widely used deep learning library that provides a rich programming interface to support various computing tasks in the field of deep learning. The framework mainly supports processor backends including multi-core CPUs, many-core GPUs, and TPUs, a dedicated chip for machine learning. The TensorFlow framework also accelerates the core computing of processor backends such as CPUs, GPUs, and TPUs by calling high-performance operator libraries, but lacks support for domestic heterogeneous many-core processors. How to efficiently utilize the multiple core computing acceleration libraries on the heterogeneous many-core architecture to achieve performance improvement of the TensorFlow framework is a challenge.

[0004] TensorFlow's native code lacks support for domestically produced heterogeneous many-core processors. Simply porting the code fails to fully utilize the computing power of heterogeneous many-core architectures. The typical approach to improving deep learning performance is to directly call various many-core acceleration libraries within the core TensorFlow code, but this approach can be quite complex.

[0005] First, due to the complexity of the TensorFlow framework, directly calling the acceleration library increases the difficulty of code modification and may cause many unpredictable problems by affecting the framework's default process. This approach cannot effectively separate platform-independent and platform-dependent code. Second, due to the polymorphism of the many-core acceleration library, multiple adaptability situations must be considered when calling the acceleration library. In this case, independent code logic is required to perform characteristic analysis of the core calculation and adaptive calling of the acceleration core. Summary of the Invention

[0006] The purpose of this invention is to provide a TensorFlow framework core computing acceleration engine for domestic heterogeneous many-core processors, which can give full play to the powerful computing capabilities of domestic heterogeneous many-core processors and provide automated core acceleration capabilities for deep learning tasks.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is to provide a TensorFlow framework core computing acceleration engine for domestic heterogeneous many-core processors, which is independently set between the TensorFlow framework and multiple many-core computing acceleration libraries and includes the following functional modules:

[0008] The operator interface module provides a series of standardized operator call interfaces for modifying the kernel layer code in the TensorFlow framework so that operators that need to be accelerated can call them, thereby transferring the operation process to the core computing acceleration engine;

[0009] The analysis and scheduling module analyzes operator types and operator parameter characteristics, encapsulates operator parameters into parameter structures for various many-core computing acceleration libraries, and calls different many-core computing acceleration libraries based on operator types. When multiple operators of the same operator type are implemented, the optimal implementation is automatically selected and called based on the characteristics of the operator parameters.

[0010] A debugging module, used to locate operator errors, further comprising an operator control unit, a non-number location unit, and an operator log unit;

[0011] The operator control unit turns on or off the calculation process of each operator through environment variables. If an operator is turned off, the operator calculation is performed through the default process of the TensorFlow framework. The debugging module can quickly locate the correctness problems of each operator.

[0012] The non-number locating unit detects whether a non-number is generated by checking the code after the operator calculation starts and ends, thereby locating the first scene where the non-number is generated;

[0013] Operator log unit, used to output all operator lists and parameters during model operation;

[0014] The performance analysis module is used to record the running time of each operator and analyze the performance of the model inference or training process from the operator level.

[0015] The further improved scheme in the above technical scheme is as follows:

[0016] 1. In the above solution, the debugging module configures an environment variable to shut down a certain core calculation, so that the core calculation acceleration engine no longer calls the multi-core optimization implementation of the core calculation, but instead switches to the default calculation process of the TensorFlow framework.

[0017] 2. In the above solution, the debugging module is configured with an environment variable for locating non-numbers, so that if a non-number is detected in the input or output of an operator during operation, an error site log can be output.

[0018] 3. In the above solution, the debugging module is configured with environment variables for recording operator logs, recording the operator types and parameters called during operation in files.

[0019] 4. In the above solution, the performance analysis module is configured with performance analysis environment variables, records the running time of each core calculation by timing, and generates a performance analysis log.

[0020] 5. In the above solution, it is compiled into an independent dynamic library.

[0021] Due to the application of the above technical solution, the present invention has the following advantages compared with the prior art:

[0022] This invention abstracts and strips away the core computing acceleration code related to the back-end platform, and designs a TensorFlow framework core computing acceleration engine for domestic heterogeneous many-core processors. It strips away the hardware-related and hardware-independent codes implemented by the TensorFlow core computing, and only requires a small amount of modification to the native code of the TensorFlow framework, reducing the occurrence of potential errors. While keeping the original programming interface of the framework unchanged, it hides the underlying hardware details from the user, realizes automatic adaptive calling of multiple many-core acceleration cores, gives full play to the powerful computing power of domestic heterogeneous many-cores, provides automatic core acceleration capabilities for deep learning tasks, and supports debugging, tuning and performance analysis functions for core computing, thereby reducing the difficulty of porting the TensorFlow framework to domestic heterogeneous many-core processors, providing automatic core acceleration capabilities for deep learning computing tasks based on the TensorFlow framework, and improving the debuggability and usability of the TensorFlow framework on heterogeneous many-core architectures. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Attachment Figure 1 This is the overall architecture diagram of the core computing acceleration engine of the present invention. DETAILED DESCRIPTION

[0024] Example: The present invention provides a TensorFlow framework core computing acceleration engine for domestic heterogeneous many-core processors, which is independently set between the TensorFlow framework and multiple many-core computing acceleration libraries, including the following functional modules:

[0025] The operator interface module provides a series of standardized operator call interfaces for modifying the kernel layer code in the TensorFlow framework so that operators that need to be accelerated can call them, thereby transferring the operation process to the core computing acceleration engine;

[0026] The analysis and scheduling module analyzes operator types and operator parameter characteristics, encapsulates operator parameters into parameter structures for various many-core computing acceleration libraries, and calls different many-core computing acceleration libraries based on operator types. When multiple operators of the same operator type are implemented, the optimal implementation is automatically selected and called based on the characteristics of the operator parameters.

[0027] A debugging module, used to locate operator errors, further comprising an operator control unit, a non-number location unit, and an operator log unit;

[0028] The operator control unit turns on or off the calculation process of each operator through environment variables. If an operator is turned off, the operator calculation is performed through the default process of the TensorFlow framework. The debugging module can quickly locate the correctness problems of each operator.

[0029] The non-number locating unit detects whether a non-number is generated by checking the code after the operator calculation starts and ends, thereby locating the first scene where the non-number is generated;

[0030] Operator log unit, used to output all operator lists and parameters during model operation;

[0031] The performance analysis module is used to record the running time of each operator and analyze the performance of the model inference or training process from the operator level;

[0032] The core computing acceleration engine is independent software that sits between the TensorFlow framework and the many-core computing acceleration library. It connects to the TensoFlow framework through the operator interface module and to the many-core acceleration library through the analysis and scheduling module. The debugging module and performance analysis module are its key components. The former helps users locate operator errors and understand operator details, while the latter is used for operator performance analysis, providing optimization references for users or operator developers.

[0033] The above embodiment is further explained as follows:

[0034] 1. The core computing acceleration engine provides standardized operator interfaces. By modifying the kernel layer code in the TensorFlow framework, these interfaces are called in the implementation of operators that need to be accelerated, thereby transferring the running process to the core computing acceleration engine.

[0035] 2. The core computing acceleration engine analyzes operator types and operator parameter characteristics, encapsulates operator parameters into parameter structures for each many-core acceleration library, and calls different many-core computing acceleration libraries based on operator types. When the same operator type has multiple many-core implementations, the engine automatically selects the optimal implementation based on the operator parameter characteristics and calls it.

[0036] 3. Support debugging through environment variables: If an environment variable is configured to disable a core calculation, the core calculation acceleration engine will no longer call the multi-core optimization implementation of the calculation, but will switch to the default calculation process of the TensorFlow framework to verify the correctness of each core calculation. If an environment variable is configured for debugging printing, debugging information will be output during the operation. If an environment variable is configured to record the detailed process of a core calculation, the input, output, and intermediate results of the core calculation will be recorded in the form of files.

[0037] 4. Support performance analysis through environment variables: If a performance analysis environment variable for a core calculation is configured, the running time of the calculation is recorded through timing and a performance analysis log is generated;

[0038] 5. The core computing acceleration engine is also responsible for other non-computing tasks, such as thread library initialization, memory application, management, and deregistration, and core group computing resource monitoring, completely shielding the underlying hardware details of the heterogeneous multi-core architecture.

[0039] 6. The core computing acceleration engine will be compiled into an independent dynamic library for easy updating, maintenance and debugging.

[0040] When using the above-mentioned TensorFlow framework core computing acceleration engine for domestic heterogeneous many-core processors, it strips away the hardware-related and hardware-independent codes of the TensorFlow core computing implementation, and only requires a small amount of modification to the native code of the TensorFlow framework, reducing the occurrence of potential errors. While keeping the original programming interface of the framework unchanged, it hides the underlying hardware details from the user, and realizes automatic adaptive calling of multiple many-core acceleration cores, giving full play to the powerful computing power of domestic heterogeneous many-cores, providing automatic core acceleration capabilities for deep learning tasks, and supporting debugging, tuning and performance analysis functions for core computing, thereby reducing the difficulty of porting the TensorFlow framework to domestic heterogeneous many-core processors, providing automatic core acceleration capabilities for deep learning computing tasks based on the TensorFlow framework, and improving the debuggability and usability of the TensorFlow framework on heterogeneous many-core architectures.

[0041] To facilitate a better understanding of the present invention, the following briefly explains the terms used herein:

[0042] Domestic heterogeneous many-core processor: a high-performance heterogeneous central processing unit that integrates a small number of general-purpose main cores that undertake management, communication and computing functions and a large number of streamlined slave cores that undertake computing functions on a complete chip; the general-purpose main core runs a general operating system, mainly undertakes the management and control functions of the entire chip, and also undertakes certain computing functions and the communication function between the chip and the outside world; the streamlined slave core plays the role of accelerating computing.

[0043] TensorFlow framework: A symbolic mathematical system based on data flow programming, widely used in the programming implementation of various machine learning algorithms. It has a multi-layered structure and can be deployed on various servers, PC terminals and web pages, and supports high-performance numerical computing on CPU, GPU and TPU platforms.

[0044] Core Computing (OP): The TensorFlow framework reconstructs neural network models or other computing tasks into computational graphs consisting of core computing (OP). The efficiency of core computing plays a vital role in the overall efficiency of deep learning applications.

[0045] The above embodiments are intended only to illustrate the technical concepts and features of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. They are not intended to limit the scope of protection of the present invention. Any equivalent changes or modifications made in accordance with the spirit of the present invention are intended to be covered by the scope of protection of the present invention.

Claims

1. A TensorFlow framework core computing acceleration engine for domestic heterogeneous many-core processors, characterized by: It is independently set between the TensorFlow framework and multiple many-core computing acceleration libraries, and includes the following functional modules: The operator interface module provides a series of standardized operator call interfaces for modifying the kernel layer code in the TensorFlow framework so that operators that need to be accelerated can call them, thereby transferring the operation process to the core computing acceleration engine; The analysis and scheduling module analyzes operator types and operator parameter characteristics, encapsulates operator parameters into parameter structures for various many-core computing acceleration libraries, and calls different many-core computing acceleration libraries based on operator types. When multiple operators of the same operator type are implemented, the optimal implementation is automatically selected and called based on the characteristics of the operator parameters. A debugging module, used to locate operator errors, further comprising an operator control unit, a non-number location unit, and an operator log unit; The operator control unit opens or closes the calculation process of each operator through environment variables. If an operator is closed, the closed operator is calculated through the default process of the TensorFlow framework. The debugging module can quickly locate the correctness problems of each operator. The non-number locating unit detects whether a non-number is generated by checking the code after the operator calculation starts and ends, thereby locating the first scene where the non-number is generated; Operator log unit, used to output all operator lists and parameters during model operation; The performance analysis module is used to record the running time of each operator and analyze the performance of the model inference or training process from the operator level.

2. The TensorFlow framework core computing acceleration engine for domestic heterogeneous many-core processors according to claim 1 is characterized by: The debugging module configures the environment variables to shut down the core calculation, so that the core calculation acceleration engine no longer calls the multi-core optimization implementation of the core calculation, but instead switches to the default calculation process of the TensorFlow framework.

3. The TensorFlow framework core computing acceleration engine for domestic heterogeneous many-core processors according to claim 1, characterized in that: The debugging module is configured with an environment variable for locating non-numbers, so that if a non-number is detected in the input or output of an operator during operation, an error site log can be output.

4. The TensorFlow framework core computing acceleration engine for domestic heterogeneous many-core processors according to claim 1, characterized in that: The debugging module is configured with environment variables for recording operator logs, and records the operator types and parameters called during the operation in the form of files.

5. The TensorFlow framework core computing acceleration engine for domestic heterogeneous many-core processors according to claim 1, characterized in that: The performance analysis module is configured with performance analysis environment variables, records the running time of each core calculation by timing, and generates a performance analysis log.

6. The TensorFlow framework core computing acceleration engine for domestic heterogeneous many-core processors according to claim 1, characterized in that: Compiled into a separate dynamic library.

Images