Method for testing rear end of third-party equipment by using PyTorch test set and related product thereof

Abstract

The invention relates to a method for testing the rear end of third-party equipment by utilizing a PyTorch test set and related products thereof. The related products comprise equipment, a device, a board card and a computer readable storage medium. The apparatus may be included in a computing processing device of a combined processing device, which may include one or more data processing devices. The combined processing device can also comprise an interface device and other processing devices. And the computing processing device interacts with other processing devices to jointly complete computing operation specified by a user. The combined processing device can further comprise a storage device, and the storage device is connected with the equipment and the other processing devices and used for storing data of the equipment and the other processing devices. Through the scheme of the invention, an original test set code does not need to be directly modified, and automatic adaptation and testing of the rear end of the third-party equipment can be realized.

Description

Technical Field

[0001] The present disclosure generally relates to the field of computers. More specifically, the present disclosure relates to a method, device, computer-readable storage medium, and board for testing a third-party device backend using a PyTorch test set. Background Art

[0002] PyTorch, the Python version of torch, is an open-source artificial intelligence framework developed by Meta. It mainly accelerates deep neural networks through GPUs. With the rapid development of the artificial intelligence chip industry, the types of chips that can be used for deep neural networks are increasing. In order to better support different types of artificial intelligence chips, the PyTorch community provides mechanisms such as cppExtension and privateUseKey to facilitate the access of third-party chips. However, for its CI (continuous integration) test set, it still focuses on testing the GPU backend and lacks comprehensive testing of third-party chip backends.

[0003] In order to fully support the various functions of PyTorch and improve the quality of PyTorch integration in the chip backend, third-party chip backend manufacturers have to modify the PyTorch CI test set to maximize testing and ensure the integrity of third-party chip backend support for PyTorch features. However, there are two main problems with this modification: first, the number of test cases involved in the modification is huge, which is prone to errors and omissions; second, due to the frequent updates and iterations of the PyTorch community version, there is a lot of repetitive work and a huge time-consuming process for modifications to different versions, making it impossible to verify the chip backend functions in a timely manner. These problems have seriously affected the efficiency and quality of third-party chip backend adaptation to PyTorch.

[0004] In view of this, there is an urgent need in the prior art for a method that can automatically adapt to the PyTorch test set, so that the third-party chip backend can run the PyTorch community CI test set without modifying the original test set code, and can automatically adapt to changes in the test sets of various PyTorch versions. This method should be able to significantly reduce the workload of third-party chip manufacturers, improve testing efficiency, and ensure the comprehensiveness and timeliness of testing. Summary of the invention

[0005] In view of the technical problems mentioned in the above background technology section, the present disclosure proposes a solution for testing the backend of third-party devices using the PyTorch test set. The technical solution of the present disclosure can effectively solve the problems faced by third-party chip manufacturers when adapting the PyTorch test set, such as large modification workload, easy errors, and repeated work caused by frequent version updates.

[0006] In a first aspect, the present disclosure provides a method for testing a third-party device backend using a PyTorch test set, comprising: loading a pytest plug-in for the PyTorch test set; dynamically replacing the PyTorch test set using the pytest plug-in and Python's monkey patch mechanism; collecting replaced test cases; and running the test cases using the third-party device backend for testing.

[0007] In a second aspect, the present disclosure provides a device for testing a third-party device backend using a PyTorch test set, including: a loading module for loading a pytest plug-in for the PyTorch test set; a replacement module for dynamically replacing the PyTorch test set using the pytest plug-in and Python's monkey patch mechanism; a collection module for collecting replaced test cases; and a running module for running the test cases using a third-party device backend for testing.

[0008] In a third aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0009] In a fourth aspect, the present disclosure provides an apparatus for testing a third-party device backend using a PyTorch test set, comprising: a processor; a memory for storing a computer program for testing a third-party device backend using a PyTorch test set; wherein the processor is used to execute the computer program to implement the method described in the first aspect.

[0010] In a fifth aspect, the present disclosure provides a board for testing a third-party device backend using a PyTorch test set, comprising the device described in the second aspect or the apparatus described in the fourth aspect.

[0011] Through the solutions provided in the above aspects of the present disclosure, it is possible to automatically adapt the PyTorch test set and efficiently test the third-party device backend. Specifically, by loading the pytest plug-in for the PyTorch test set and dynamically replacing the test set using Python's monkey patch mechanism, it is possible to automatically adapt and test the third-party device backend without directly modifying the original test set code.

[0012] In some embodiments, the PyTorch test set is replaced according to the specific backend settings, custom interfaces, and adaptation test cases of the third-party device backend, which can fully cover the characteristics of the third-party device backend and improve the comprehensiveness and accuracy of the test. In other embodiments, by parsing the backend information in the PyTorch interface input and replacing some or all of the default settings, the parts that need to be replaced can be accurately located to improve the accuracy and efficiency of the replacement. In some embodiments, the backend information in the PyTorch interface input can be parsed by Python decorators to replace some or all of the default settings.

[0013] In some embodiments, by using Python decorators to identify torch.device objects and extract backend type information, flexible replacement of different types of backend settings can be achieved. In other embodiments, by identifying the custom interface of the third-party device backend and replacing the corresponding PyTorch interface, the unique functions of the third-party device backend can be fully utilized to improve the pertinence and comprehensiveness of the test.

[0014] In some embodiments, by identifying and modifying test cases that have functional restrictions on the backend of a third-party device, the compatibility of the test cases with the backend of the third-party device can be ensured, thereby improving the effectiveness and comprehensiveness of the test. In addition, the present disclosure also provides corresponding devices, storage media, and artificial intelligence acceleration boards, which can conveniently implement the above method and improve the automation and efficiency of the test. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present disclosure will become readily understood. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0016] Figure 1 is a structural diagram showing a board according to an embodiment of the present disclosure;

[0017] Figure 2 is a structural diagram showing a combined processing device according to an embodiment of the present disclosure;

[0018] Figure 3 is a schematic diagram showing the internal structure of a computing device according to an embodiment of the present disclosure;

[0019] Figure 4 is a schematic diagram showing the internal structure of a processor core according to an embodiment of the present disclosure;

[0020] Figure 5 is a schematic diagram showing a data writing process between processor cores of different clusters according to an embodiment of the present disclosure;

[0021] Figure 6 is a structural diagram showing the software and hardware architecture of data flow programming according to an embodiment of the present disclosure;

[0022] Figure 7 is a simplified flowchart schematically illustrating a method for testing a third-party device backend using a PyTorch test set according to an embodiment of the present disclosure;

[0023] Figure 8 A detailed flowchart of a method for testing a third-party device backend using a PyTorch test set according to an embodiment of the present disclosure is schematically shown;

[0024] Fig. 9 A structural block diagram of a device for testing a third-party device backend using a PyTorch test set according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0026] It should be understood that the terms "first", "second", "third", "fourth", etc. in the claims, specifications and drawings of the present disclosure are used to distinguish different objects rather than to describe a specific order. The terms "include" and "comprise" used in the specifications and claims of the present disclosure indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0027] It should also be understood that the terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. As used in this disclosure and claims, the singular forms of "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in this disclosure and claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations.

[0028] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0029] As mentioned earlier, PyTorch, as an open source deep learning framework from Meta, mainly uses GPU to accelerate neural networks. However, with the booming development of the artificial intelligence chip industry, various chips that can be used for deep learning have emerged in an endless stream. In order to better support these third-party AI chips, although the PyTorch community provides mechanisms such as cppExtension and privateUseKey to facilitate chip manufacturers to adapt the backend, its CI test set still focuses mainly on the testing of the GPU backend, and the testing coverage of the third-party chip backend is insufficient. This has led to third-party chip manufacturers having to make a lot of modifications to the test set to cover the characteristics of their chip backend when adapting PyTorch, and the frequent iterations of PyTorch versions have made the related modification work repetitive and cumbersome, time-consuming, and difficult to conduct timely and comprehensive verification of the chip backend functions.

[0030] In view of this, the present disclosure proposes a method, equipment, device and board for testing the backend of a third-party device using a PyTorch test set. The method uses the plug-in mechanism of pytest and the monkey patch technology of Python to realize the adaptive modification of the PyTorch test set. Specifically, by loading the pytest plug-in, the corresponding backend settings, interfaces and test cases in the test set are dynamically replaced using the monkey patch mechanism, so that the third-party device backend can run the PyTorch test set without modifying the original test set code; by collecting the replaced test cases and running them on the backend of the third-party device, the functional integrity of the third-party device adaptation to PyTorch can be fully evaluated. The present disclosure also proposes corresponding test equipment, devices and boards, which can be conveniently implemented on various chip platforms by setting functional modules such as loading, replacing, collecting and running, and equipped with a memory for storing the adaptation program and a processor for executing the program. The present invention can significantly improve the test efficiency of third-party devices adapting to PyTorch and accelerate its compatible adaptation process with different versions of PyTorch.

[0031] It should be understood that the third-party device in the present disclosure may be various processing devices such as chips and boards, and the description of the present disclosure mainly uses chips or AI chips as examples.

[0032] Figure 1 FIG. 1 is a schematic diagram showing the structure of a board 100 according to an embodiment of the present disclosure. Figure 1 As shown, the board 100 includes a chip (or "processing chip") 101, which is a system-on-chip (SoC), or system on chip, integrated with one or more combined processing devices, which are artificial intelligence computing units that support various deep learning and machine learning algorithms to meet the intelligent processing needs in complex scenarios in the fields of computer vision, speech, natural language processing, data mining, etc. In particular, deep learning technology is widely used in the field of cloud intelligence. A notable feature of cloud intelligence applications is the large amount of input data, which places high demands on the storage capacity and computing power of the platform. The board 100 of this embodiment is suitable for cloud intelligence applications and has huge off-chip storage, on-chip storage, and a large amount of computing power.

[0033] The chip 101 is connected to the external device 103 through the external interface device 102. The external device 103 is, for example, a server, a computer, a camera, a display, a mouse, a keyboard, a network card or a WIFI interface. The data to be processed can be transmitted from the external device 103 to the chip 101 through the external interface device 102. The calculation result of the chip 101 can be transmitted back to the external device 103 via the external interface device 102. According to different application scenarios, the external interface device 102 can have different interface forms, such as a PCIe interface.

[0034] The board 100 also includes a storage device 104 for storing data, which includes one or more storage units 105. The storage device 104 is connected to the control device 106 and the chip 101 through a bus and transmits data. The control device 106 in the board 100 is configured to control the state of the chip 101. To this end, in an application scenario, the control device 106 may include a microcontroller (MCU).

[0035] Figure 2 1 is a block diagram showing the structure of the combined processing device 200 in the chip 101 of this embodiment. Figure 2 As shown in , the combined processing device 200 includes a computing device 201 , an interface device 202 , a processing device 203 and a DRAM 204 .

[0036] The computing device 201 is configured to execute user-specified operations, and is mainly implemented as a single-core intelligent processor or a multi-core intelligent processor to perform deep learning or machine learning calculations. It can interact with the processing device 203 through the interface device 202 to jointly complete the user-specified operations.

[0037] The interface device 202 is used to transmit data and control instructions between the computing device 201 and the processing device 203. For example, the computing device 201 can obtain input data from the processing device 203 via the interface device 202 and write it into the storage device on the computing device 201 chip. Further, the computing device 201 can obtain control instructions from the processing device 203 via the interface device 202 and write them into the control cache on the computing device 201 chip. Alternatively or optionally, the interface device 202 can also read data in the storage device of the computing device 201 and transmit it to the processing device 203.

[0038] The processing device 203, as a general processing device, performs basic controls including but not limited to data handling, starting and / or stopping the computing device 201. Depending on the implementation, the processing device 203 can be a central processing unit (CPU), a graphics processing unit (GPU) or one or more types of processors in other general and / or special processors, these processors include but are not limited to digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, only with respect to the computing device 201 disclosed in the present invention, it can be regarded as having a single-core structure or a homogeneous multi-core structure. However, when the computing device 201 and the processing device 203 are integrated and considered together, the two are regarded as forming a heterogeneous multi-core structure.

[0039] DRAM 204 is used to store data to be processed, and is a DDR memory, which is usually 16G or larger in size and is used to store data of the computing device 201 and / or the processing device 203 .

[0040] Figure 3 The internal structure diagram of the computing device 201 is shown. The computing device 201 is used to process input data such as computer vision, speech, natural language, and data mining. The computing device 201 in the figure adopts a multi-core hierarchical structure design. The computing device 201 is a system on a chip, which includes multiple clusters, and each cluster includes multiple processor cores, which can be used to execute the tasks issued by the present disclosure. In other words, the computing device 201 is composed of a hierarchy of system on a chip-cluster-processor core.

[0041] At the system-on-chip level, Figure 3 As shown, the computing device 201 includes an external storage controller 301 , a peripheral communication module 302 , an on-chip interconnect module 303 , a synchronization module 304 , and multiple clusters 305 .

[0042] There may be multiple external storage controllers 301, two of which are shown in the figure as an example, and are used to respond to access requests issued by the processor core and access external storage devices, such as Figure 2 DRAM 204 in the chip, thereby reading data from outside the chip or writing data. The peripheral communication module 302 is used to receive control signals from the processing device 203 through the interface device 202 to start the computing device 201 to perform tasks. The on-chip interconnect module 303 connects the external storage controller 301, the peripheral communication module 302 and multiple clusters 305 to transmit data and control signals between each module. The synchronization module 304 is a global synchronization barrier controller (Global Barrier Controller, GBC), which is used to coordinate the work progress of each cluster and ensure the synchronization of information. Multiple clusters 305 are the computing cores of the computing device 201. Four are shown as an example in the figure. With the development of hardware, the computing device 201 disclosed in the present invention can also include 8, 16, 64, or even more clusters 305.

[0043] At the cluster level, Figure 3 As shown, each cluster 305 includes multiple processor cores (IPU cores) 306 and a memory core (MEM core) 307 .

[0044] Processor core 306 Figure 3 4 are shown as an example, and the present disclosure does not limit the number of processor cores 306. Its internal architecture is as follows Figure 4 Each processor core 306 includes three modules: a control module 31 , a computing module 32 and a storage module 33 .

[0045] The control module 31 is used to coordinate and control the operation of the operation module 32 and the storage module 33 to complete the deep learning task, and includes an instruction fetch unit (IFU) 411 and an instruction decode unit (IDU) 412. The instruction fetch unit 411 is used to obtain instructions from the processing device 203, and the instruction decode unit 412 decodes the obtained instructions and sends the decoding results to the operation module 32 and the storage module 33 as control information.

[0046] The operation module 32 includes a vector operation unit 421 and a matrix operation unit 422. The vector operation unit 421 is used to perform vector operations and can support complex operations such as vector multiplication, addition, and nonlinear transformation; the matrix operation unit 422 is responsible for the core calculation of the deep learning algorithm, namely matrix multiplication and convolution.

[0047] The storage module 33 is used to store or move related data, including a neuron RAM (NRAM) 431, a weight RAM (WRAM) 432, an input / output direct memory access module (IODMA) 433, and a move direct memory access module (MVDMA) 434. NRAM 431 is used to store input and output data and intermediate results for calculation by the processor core 306; WRAM 432 is used to store the weights of the deep learning network; IODMA 433 controls the memory access between NRAM 431 / WRAM 432 and DRAM 204 through the broadcast bus 309; MVDMA 434 is used to control the memory access between NRAM 431 / WRAM 432 and SRAM 308.

[0048] Combination of the above Figure 4 The internal components and details of the processor core 306 are described. Figure 3 The storage core 307 is mainly used for storage and communication, that is, to store shared data or intermediate results between the processor cores 306, and to perform communication between the cluster 305 and the DRAM 204, communication between the clusters 305, and communication between the processor cores 306. In other embodiments, the storage core 307 has the capability of scalar operation and is used to perform scalar operation.

[0049] The storage core 307 includes a shared memory unit (SRAM) 308, a broadcast bus 309, a cluster direct memory access module (CDMA) 310, and a global direct memory access module (GDMA) 311. The SRAM 308 plays the role of a high-performance data transfer station. The data reused between different processor cores 306 in the same cluster 305 does not need to be obtained from the DRAM 204 by each processor core 306, but is transferred between the processor cores 306 via the SRAM 308. The storage core 307 only needs to quickly distribute the reused data from the SRAM 308 to multiple processor cores 306, so as to improve the efficiency of inter-core communication and greatly reduce on-chip and off-chip input / output access.

[0050] The broadcast bus 309, CDMA 310 and GDMA 311 are respectively used to perform communication between processor cores 306, communication between clusters 305 and data transmission between clusters 305 and DRAM 204. They will be described below.

[0051] The broadcast bus 309 is used to complete high-speed communication between the processor cores 306 in the cluster 305. The broadcast bus 309 of this embodiment supports inter-core communication modes including unicast, multicast and broadcast. Unicast refers to point-to-point (i.e., single processor core to single processor core) data transmission, multicast is a communication mode of transmitting a copy of data from SRAM 308 to specific processor cores 306, and broadcast is a communication mode of transmitting a copy of data from SRAM 308 to all processor cores 306, which is a special case of multicast.

[0052] The CDMA 310 is used to control the memory access of the SRAM 308 between different clusters 305 in the same computing device 201 . Figure 5 A schematic diagram is shown when a processor core wants to write data to a processor core of another cluster to illustrate the working principle of CDMA310. In this application scenario, the same computing device includes multiple clusters. For the convenience of explanation, only cluster 0 and cluster 1 are shown in the figure. Cluster 0 and cluster 1 each include multiple processor cores. For the convenience of explanation, cluster 0 in the figure only shows processor core 0, and cluster 1 only shows processor core 1. Processor core 0 wants to write data to processor core 1.

[0053] First, processor core 0 sends a unicast write request to write data into the local SRAM 0. CDMA 0 acts as the master and CDMA 1 acts as the slave. The master pushes the write request to the slave, that is, the master sends the write address AW and write data W to transfer the data to SRAM 1 of cluster 1. Then the slave sends a write response B in response. Finally, processor core 1 of cluster 1 sends a unicast read request to read the data from SRAM 1.

[0054] Back to Figure 3, GDMA 311 cooperates with external memory controller 301 to control the access of SRAM 308 of cluster 305 to DRAM 204, or reads data from DRAM 204 to SRAM 308. As can be seen from the above, the communication between DRAM 204 and NRAM 431 or WRAM 432 can be realized through two channels. The first channel is to directly connect DRAM 204 and NRAM 431 or WRAM 432 through IODAM 433; the second channel is to first transmit data between DRAM 804 and SRAM 308 through GDMA 311, and then transmit data between SRAM 308 and NRAM 431 or WRAM 432 through MVDMA 434. Although it seems that the second channel requires more components to participate and the data flow is longer, in fact, in some embodiments, the bandwidth of the second channel is much greater than that of the first channel, so the communication between DRAM 204 and NRAM 431 or WRAM 432 may be more efficient through the second channel. The embodiments of the present disclosure can select a data transmission channel according to the hardware conditions thereof.

[0055] In other embodiments, the functions of GDMA 311 and IODMA 433 can be integrated into the same component. For the convenience of description, this disclosure regards GDMA 311 and IODMA 433 as different components. For those skilled in the art, as long as the functions implemented and the technical effects achieved are similar to those of this disclosure, they belong to the protection scope of this disclosure. Furthermore, the functions of GDMA 311, IODMA 433, CDMA 310, and MVDMA 434 can also be implemented by the same component. Similarly, as long as the functions implemented and the technical effects achieved are similar to those of this disclosure, they belong to the protection scope of this disclosure.

[0056] Combination of the above Figure 1-Figure 5 The hardware architecture and internal structure of the present disclosure are described in detail. It is understandable that the above description is only exemplary and not restrictive. According to different application scenarios and hardware specifications, those skilled in the art may also make changes to the board (or artificial intelligence device) and its internal structure of the present disclosure, and these changes still fall within the scope of protection of the present disclosure. Figure 1-Figure 5 In addition to the hardware architecture shown, the solution of the present disclosure also relates to a hardware and software architecture, which will be described below.

[0057] Figure 6The design diagram of the hardware and software architecture in one embodiment of the present disclosure is shown. As shown in the figure, the hardware and software architecture in this embodiment may include an AI processor 601, a driver and operating system 602, a compiler and programming language 603, a library 604, a framework layer 605, and an application layer 606. It is understandable that the hardware and software architecture here can be applied to the artificial intelligence computing system or heterogeneous computing platform of this application.

[0058] Specifically, the AI ​​processor 601 (which may be included in the board described below in conjunction with the accompanying drawings, for example) considers both operation optimization and data handling optimization in hardware design. To this end, it uses a customized operation unit to accelerate operations, and uses on-chip storage to accelerate data handling, thereby obtaining extremely high performance and energy efficiency. In addition, in order to support various algorithm optimizations, the AI ​​processor 601 may have a customized operation unit and instruction set, wherein the instruction set may provide operation instructions (scalar, vector and / or matrix) of different granularities. Further, when considering factors such as algorithm memory access characteristics, hardware cost, verification difficulty, etc., on-chip storage may be used, and data handling may be optimized. In actual operation, the AI ​​processor disclosed in the present invention can achieve a speed that is more than dozens of times faster than that of a mainstream GPU (graphics processing unit).

[0059] The driver and operating system 602 are mainly responsible for implementing the scheduling of tasks on the AI ​​processor 601. The scheduling operation can, for example, implement scheduling according to task priority, communication and synchronization between multiple devices, etc. For the compiled program, it can implement the scheduling and execution of the tasks to be implemented on a specific processor through the operating system and the driver, including but not limited to the following operations: allocating and releasing device memory, implementing data transmission between devices, maintaining task queues, and scheduling tasks according to priority to achieve synchronization and collaboration between multiple devices.

[0060] The compiler and programming language 603 may be a set of assembly languages ​​developed for the instruction set of the AI ​​processor 601. In applications, it may translate deep learning operators developed for the AI ​​processor 601 into processor instruction combinations to facilitate calling the AI ​​processor 601, thereby efficiently using the AI ​​processor 601. In some application scenarios, the compiler may be used to perform the intermediate expression stage of compilation to optimize compilation.

[0061] The library 604 may include a runtime library 614 and a machine learning library 624. In one implementation scenario, the aforementioned library 604 may use the instruction set of the AI ​​processor 601 and partially optimize according to the instruction set of the AI ​​processor 601 to improve the running speed of the operator. The runtime library 614 may be a set of high-performance operator libraries specially developed for the AI ​​processor 601, and it may be used to complete the interaction between the general processor and the artificial intelligence processor. Further, the runtime library 614 may also provide a set of interfaces for artificial intelligence processors. For the machine learning library 624, it may be used to accelerate various machine learning or deep learning algorithms on the artificial intelligence processor. Specifically, the machine learning library 624 may provide a set of efficient, general, flexible and scalable programming interfaces, and the upper-level machine learning applications may directly adopt the programming interfaces of various programming frameworks (such as pytorch, TensorFlow, Caffe, MXNet, etc.), or may use the interfaces provided by the machine learning library 624 to directly program. In addition, the machine learning library 624 of the present disclosure may facilitate the calling of the hardware platform, and the runtime library 614 may implement some basic commonly used operators, such as various operations such as convolution and pooling.

[0062] The framework layer 605 can add encapsulation of operators developed for AI processors, and mainly encapsulates operators of the runtime library 614. In addition, the framework layer 605 can also modify related task scheduling or memory management. In an application scenario, the framework layer 605 can adopt the architecture of frameworks such as Pytorch and TensorFlow.

[0063] The specific embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0064] Figure 7 Schematically illustrates a simplified flowchart of a method 700 for testing a third-party device backend using a PyTorch test set according to an embodiment of the present disclosure.

[0065] like Figure 7As shown, at step S702, the pytest plug-in for the PyTorch test set is loaded. As known to those skilled in the art, the PyTorch test set is a collection of a series of test cases for testing the functions and performance of the PyTorch framework, which includes various types such as unit tests, integration tests, and end-to-end tests, covering most of the functional modules of PyTorch, and is an important tool for verifying the correctness and reliability of the PyTorch framework. Furthermore, the pytest plug-in is an important part of the pytest test framework, which can realize the expansion and customization of test functions. By introducing the pytest plug-in customized for the PyTorch test set, the test set can be given the ability to execute on third-party AI chips without modifying the original test script of PyTorch.

[0066] Then, at step S704, the PyTorch test set is dynamically replaced by using the monkey patch mechanism of Python through the pytest plug-in. In the scenario disclosed herein, the monkey patch mechanism refers to modifying and replacing existing code at runtime without modifying the original code, which is a technical means of dynamically changing program behavior. It takes advantage of the dynamic characteristics and flexibility of the Python language to achieve the purpose of inserting custom logic by replacing functions, methods or attributes at runtime. According to the application scenario disclosed herein, the dynamic replacement here can be to modify the part involving chips and operators in the PyTorch test set, replace the original default CPU or CUDA device with a third-party AI chip, replace the native operator with an operator with the same function optimized and implemented on a third-party AI chip, etc., so that the test code can be correctly executed on a third-party AI chip.

[0067] Next, at step S706, PyTorch test cases that have been dynamically replaced by monkey patches are collected to obtain a test set adapted to third-party AI chips, in preparation for subsequent test execution.

[0068] Finally, in step S708, a third-party AI chip can be used as the device backend to run the dynamically replaced and collected PyTorch test cases to implement the adaptability test and evaluation of the third-party AI chip. This step actually adapts and integrates the third-party AI chip with the PyTorch framework. By executing a series of tests, the third-party AI chip is evaluated for its support for the various functional features of PyTorch, and it is verified whether it can complete various machine learning tasks stably and efficiently. At the same time, the problems and failed cases encountered during the test also provide important clues for subsequent adaptation and optimization.

[0069] Through the above method 700 disclosed in the present invention, the test efficiency and coverage of third-party AI chips adapted to the PyTorch framework can be significantly improved. Traditional adaptation testing requires manual modification of a large amount of test code for each PyTorch version, which is time-consuming, labor-intensive and easy to miss. The method disclosed in the present invention utilizes the pytest plug-in and monkey patch mechanism to realize automatic adaptation and dynamic replacement of test sets, so that third-party AI chips can reuse PyTorch's original massive test cases and continue to follow up on the update of PyTorch versions, which greatly saves the workload of test development and improves the flexibility and maintainability of the test.

[0070] Figure 8 A detailed flowchart of method 800 for testing a third-party device backend using a PyTorch test set according to an embodiment of the present disclosure is schematically shown.

[0071] like Figure 8 As shown, the process starts at step S802, and at step S804, the pytest plug-in for the PyTorch test set is loaded. The PyTorch test set is a set of test cases officially provided by PyTorch for testing the functions and performance of the PyTorch framework, covering the main functional modules of PyTorch such as tensor operations, data loading, neural network layers, GPU acceleration, etc. Pytest is a mature Python testing framework that supports a flexible plug-in mechanism. The present invention customizes and develops a pytest plug-in for the PyTorch test set, so that the PyTorch test set can be easily run on third-party devices without modifying the PyTorch source code and test scripts.

[0072] Next, the PyTorch test set is dynamically replaced by the pytest plug-in using Python's monkey patch mechanism. Specifically, the dynamic replacement may include the following three aspects. First, in step S806, the PyTorch test set is replaced according to the specific backend settings of the third-party device backend. Third-party devices usually have their own unique hardware characteristics and software stacks, and require specific environmental configurations and parameters to exert their performance advantages. The backend settings here refer to these configurations and parameters.

[0073] To implement step S806, the backend information in the PyTorch interface input can be first parsed. Each functional interface of PyTorch usually contains a device parameter, which is used to specify which device (such as CPU or GPU) the operation is executed on. Here, the device parameter is analyzed to extract the information about the device backend contained therein.

[0074] As an example, the torch.device object can be identified through the Python decorator. torch.device is a class in PyTorch used to abstract and manage different devices. By analyzing the type of the torch.device object (such as "cpu" or "cuda"), the backend used by the current PyTorch operation can be accurately located. Next, the backend type information in the torch.device object can be extracted. Using the torch.device object identified in the previous step, read its device_type attribute to obtain the type of the current backend, such as "cpu", "cuda", "xla", etc.

[0075] After that, some or all of the default settings in the backend information can be replaced with the specific backend settings of the third-party chip backend. Some or all of the default backend types extracted in the above steps can be replaced with the backend type identifiers corresponding to the third-party chip, for example, "cuda" can be replaced with "mlu", etc. At the same time, other backend configuration parameters can also be customized to match the optimal working parameters of the third-party device.

[0076] In step S808, the PyTorch test set is replaced according to the custom interface of the third-party device backend. In order to fully utilize the chip acceleration capabilities in PyTorch, third-party chip manufacturers usually develop some dedicated PyTorch extension interfaces, which are functionally equivalent to the native PyTorch interfaces, but are optimized for the chip in implementation; for example, in order to reflect the different attributes of the third-party device from other devices, some custom interfaces will be set accordingly.

[0077] To this end, the custom interface of the third-party chip backend can be identified first. For example, the PyTorch test script can be analyzed to find the PyTorch interface called therein, and the adaptation document of the third-party chip can be consulted to determine the part that can be replaced with the custom interface of the third-party chip. Then, the PyTorch interface with the same or similar function as the custom interface can be replaced with the custom interface. It can be seen that by using Python's monkey patch technology, the native PyTorch interface call can be replaced with the equivalent custom interface of the third-party chip without modifying the original PyTorch test script, so as to take advantage of the acceleration advantages of the third-party chip.

[0078] In step S810, the PyTorch test set is replaced according to the adaptation test case of the third-party device backend. Due to the particularity of the third-party device backend (such as AI chip), it is likely that it will not be fully compatible with some test cases of PyTorch. In order to run the test on the third-party chip smoothly, it is necessary to customize this part of the limited test cases.

[0079] To this end, test cases that have functional restrictions on the backend of third-party devices can be identified. For example, by analyzing the adaptation documents of third-party chips, summarizing their known incompatible features, and finding test cases in the PyTorch test set that may cause compatibility issues. Then, modify the test cases to adapt to the functional characteristics of the third-party chip backend. Specifically, the problem test cases identified above are customized to bypass the incompatible characteristics of the chip, or downgrade the test requirements to ensure the smooth execution of the test on the third-party chip platform. Thereafter, the modified adaptation test case can be output.

[0080] It can be seen that the above-mentioned replacement steps S806, S808 and S810 of the present disclosure are automatically executed through the pytest plug-in and monkeypatch mechanism. The whole process does not require manual modification of the PyTorch source code and test scripts, which greatly improves the automation and reusability of the test.

[0081] Next, at step S812, the replaced test cases are collected to obtain a PyTorch test set that is fully adapted to the backend of the third-party device, which can be used to evaluate the adaptability and support capabilities of the third-party device for PyTorch. Thereafter, at step S814, a third-party device (such as the AI ​​chip disclosed in the present invention) is used as the backend to run the adapted PyTorch test cases collected in step S812 to comprehensively test and evaluate the integration and adaptation of the third-party device and PyTorch. This step obtains the performance data of the third-party chip running the various functions of PyTorch by actually executing the test, helping PyTorch users to objectively evaluate the chip's support capabilities for PyTorch. At the same time, the problems exposed in the test also provide a basis for chip developers to further improve and enhance adaptability. Finally, the process ends at step S816.

[0082] It should be understood that in other implementations, the PyTorch test set can also be dynamically replaced according to at least one of the backend settings, custom interfaces, and adaptation test cases of the third-party device to match different third-party devices or different application scenarios.

[0083] The above method disclosed in the present invention makes full use of the pytest plug-in mechanism and the dynamic characteristics of the Python language. Through flexible automated scripts, it realizes the adaptive transformation of the PyTorch test set to third-party AI chips, so that a large number of PyTorch test cases can be directly run on third-party AI chips without manual modification, which greatly reduces the test development workload of chip manufacturers, improves test efficiency and coverage, accelerates the adaptation progress of chips to PyTorch, and effectively promotes the integrated development of AI chips and PyTorch ecology.

[0084] Fig. 9 The structural block diagram of a device 900 for testing a third-party device backend using a PyTorch test set according to an embodiment of the present disclosure is schematically shown.

[0085] like Fig. 9 As shown, the device 900 provided in the present application for testing the backend of a third-party device using a PyTorch test set may be a chip 101 integrated with a combined processing device 200 as described in the embodiment of the present application, or a computing device including the chip such as a server, a PC, or an embedded system. The device 900 may include a loading module 902, a replacement module 904, a collection module 906, and an operation module 908, and the above modules may be implemented collaboratively through the combined processing device 200 in the chip 101.

[0086] Specifically, the loading module 902 is used to load the pytest plug-in for the PyTorch test set. The plug-in is customized and developed for the PyTorch test set by the present disclosure. It uses the flexible plug-in mechanism of pytest to enable the PyTorch test set to run on a third-party device backend (such as an AI chip) without changing the PyTorch source code and test scripts. The loading module 902 can use the instructions and data storage units in the processing device 203 of the chip 101 to realize the loading of the pytest plug-in.

[0087] Further, the replacement module 904 is used to dynamically replace the PyTorch test set through the above-mentioned pytest plug-in and the monkey patch mechanism of Python to adapt to the third-party device backend. The replacement content may include: replacement of backend configuration parameters optimized for third-party chips, replacement of custom interfaces of third-party chips, and replacement of test cases modified for chip characteristics. The replacement module 904 can use the processing device 203 of the chip 101 to implement monkey patch rewriting of the test set through the Python interpreter and JIT (Just-in-time) compiler.

[0088] The collection module 906 is used to collect PyTorch test cases adapted to third-party devices after being processed by the replacement module 904, and organize them into test sets that can be run on third-party devices. The collection module 906 can use the code static analysis and dependency management tools in the processing device 203 of the chip 101 to identify the replaced test cases and package them. Finally, the operation module 908 is used to execute the modified PyTorch test set output by the collection module 906 in an environment where the third-party AI chip is used as the backend, and collect the test results to automatically evaluate the adaptation performance of the third-party chip to PyTorch. As an example, the operation module 908 can use the computing device 201 of the chip 101 to utilize its multi-core collaborative computing power to efficiently run large-scale test cases.

[0089] The above four functional modules can work closely together through the on-chip interconnect network and storage subsystem inside chip 101 to jointly complete the adaptive transformation and operation of the PyTorch test set, evaluate the adaptability and support capabilities of third-party AI chips for PyTorch, and thus realize the migration of the PyTorch test set to third-party AI chips. Device 900 is aimed at third-party AI chip manufacturers and can be used as a key component of the PyTorch development tool chain for AI chips to adapt. It can also be used as a test platform to verify the adaptability of chips to PyTorch at different stages of chip development, which helps AI chips to discover and solve adaptation problems as early as possible and accelerate their integration with the PyTorch ecosystem.

[0090] In addition to the device for testing the backend of a third-party device using the PyTorch test set in the above embodiment, the present application also provides a device for testing the backend of a third-party device using the PyTorch test set. The device includes a processor and a memory, wherein the memory is used to store a computer program for adapting the PyTorch test set. The processor is used to execute the above-mentioned computer program to implement the method for testing the backend of a third-party device using the PyTorch test set described above. Specifically, the computer program can guide the processor to complete the adaptive transformation of the PyTorch test set, including loading a customized pytest plug-in, replacing the device-related code in the test set through the monkey patch mechanism, collecting the transformed test cases, and driving the third-party device to execute the test cases. The device of the present application can be a general-purpose computer, a server, an embedded device, or various forms of computing devices such as an AI accelerator card and a dedicated integrated circuit, as long as it includes a processor for program execution and a memory for program storage.

[0091] The device of the present application for testing the back-end of third-party devices using the PyTorch test set can provide a PyTorch adaptation and testing platform for third-party AI chips, verify and improve their adaptability to PyTorch at various stages of chip development, help the chips adapt to the PyTorch ecosystem as early as possible, and accelerate their commercialization process.

[0092] In some embodiments, the present application may also provide a board for testing the backend of a third-party device using the PyTorch test set. The board may include the aforementioned device for testing the backend of a third-party device using the PyTorch test set, and may also include the aforementioned device for testing the backend of a third-party device using the PyTorch test set. Since the testing method of the present application has strong versatility and adaptability, the board integrated with the device or apparatus of the present application can be conveniently used to test and evaluate the PyTorch adaptability of various third-party AI chips. Chip manufacturers can integrate the AI ​​chip to be evaluated on the board, and install the board in the expansion slot of a general-purpose computer or server, and can use the massive test cases of PyTorch to comprehensively evaluate the adaptability of the AI ​​chip to PyTorch.

[0093] The above is a detailed description of the scheme of the present disclosure in conjunction with the accompanying drawings. According to different application scenarios, the equipment or device disclosed herein may include a server, a cloud server, a server cluster, a data processing device, a robot, a computer, a printer, a scanner, a tablet computer, a smart terminal, a PC device, an Internet of Things terminal, a mobile terminal, a mobile phone, a driving recorder, a navigator, a sensor, a camera, a camera, a video camera, a projector, a watch, a headset, a mobile storage, a wearable device, a visual terminal, an automatic driving terminal, a vehicle, a household appliance, and / or a medical device. The vehicle includes an airplane, a ship, and / or a vehicle; the household appliance includes a television, an air conditioner, a microwave oven, a refrigerator, an electric rice cooker, a humidifier, a washing machine, an electric light, a gas stove, and a range hood; the medical device includes a nuclear magnetic resonance apparatus, a B-ultrasound apparatus, and / or an electrocardiograph. The equipment or device disclosed herein can also be applied to the Internet, the Internet of Things, a data center, energy, transportation, public administration, manufacturing, education, power grids, telecommunications, finance, retail, construction sites, medical treatment and other fields.

[0094] Furthermore, the devices or apparatuses disclosed herein can also be used in application scenarios related to artificial intelligence, big data and / or cloud computing, such as the cloud, edge, and terminal. In one or more embodiments, devices or apparatuses with high power consumption according to the disclosed solution can be applied to cloud devices (such as cloud servers), while devices or apparatuses with low power consumption can be applied to terminal devices and / or edge devices (such as smart phones or cameras). In one or more embodiments, the hardware information of the cloud device and the hardware information of the terminal device and / or edge device are compatible with each other, so that according to the hardware information of the terminal device and / or edge device, appropriate hardware resources can be matched from the hardware resources of the cloud device to simulate the hardware resources of the terminal device and / or edge device, so as to complete the unified management, scheduling and collaborative work of end-to-end or cloud-edge-to-end.

[0095] It should be noted that, for the purpose of simplicity, the present disclosure describes some methods and embodiments thereof as a series of actions and combinations thereof, but those skilled in the art will appreciate that the scheme of the present disclosure is not limited by the order of the actions described. Therefore, based on the disclosure or teaching of the present disclosure, those skilled in the art will appreciate that some of the steps therein may be performed in other orders or simultaneously. Further, those skilled in the art will appreciate that the embodiments described in the present disclosure may be regarded as optional embodiments, i.e., the actions or modules involved therein may not necessarily be necessary for the implementation of one or some of the schemes of the present disclosure. In addition, depending on the different schemes, the present disclosure also has different focuses on the description of some embodiments. In view of this, those skilled in the art may appreciate the parts that are not described in detail in a certain embodiment of the present disclosure, and may also refer to the relevant descriptions of other embodiments.

[0096] In terms of specific implementation, based on the disclosure and teaching of this disclosure, those skilled in the art can understand that several embodiments disclosed in this disclosure can also be implemented by other methods not disclosed herein. For example, with respect to the various units in the device or apparatus embodiments described above, this article divides them on the basis of considering logical functions, and there may be other division methods in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features or functions in a unit or component can be selectively disabled. In terms of the connection relationship between different units or components, the connection discussed in the above text in conjunction with the accompanying drawings can be a direct or indirect coupling between units or components. In some scenarios, the aforementioned direct or indirect coupling involves a communication connection using an interface, wherein the communication interface can support electrical, optical, acoustic, magnetic or other forms of signal transmission.

[0097] In the present disclosure, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units. The aforementioned components or units may be located in the same location or distributed on multiple network units. In addition, according to actual needs, some or all of the units may be selected to achieve the purpose of the scheme described in the embodiments of the present disclosure. In addition, in some scenarios, multiple units in the embodiments of the present disclosure may be integrated into one unit or each unit may exist physically separately.

[0098] In some implementation scenarios, the above-mentioned integrated unit can be implemented in the form of a software program module. If implemented in the form of a software program module and sold or used as an independent product, the integrated unit can be stored in a computer-readable memory. Based on this, when the scheme of the present disclosure is embodied in the form of a software product (such as a computer-readable storage medium), the software product can be stored in a memory, which may include several instructions to enable a computer device (such as a personal computer, a server or a network device, etc.) to perform some or all of the steps of the method described in the embodiment of the present disclosure. The aforementioned memory may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a flash drive, a read-only memory ("Read Only Memory", abbreviated as ROM), a random access memory ("Random Access Memory", abbreviated as RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0099] In some other implementation scenarios, the above-mentioned integrated units may also be implemented in the form of hardware, that is, as specific hardware circuits, which may include digital circuits and / or analog circuits. The physical implementation of the hardware structure of the circuit may include, but is not limited to, physical devices, and the physical devices may include, but are not limited to, devices such as transistors or memristors. In view of this, the various devices described herein (such as computing devices or other processing devices) may be implemented by appropriate hardware processors, such as CPUs, GPUs, FPGAs, DSPs, and ASICs. Furthermore, the aforementioned storage unit or storage device may be any appropriate storage medium (including magnetic storage medium or magneto-optical storage medium, etc.), which may be, for example, resistive random access memory ("Resistive Random Access Memory", abbreviated as RRAM), dynamic random access memory ("Dynamic Random Access Memory", abbreviated as DRAM), static random access memory ("Static Random Access Memory", abbreviated as SRAM), enhanced dynamic random access memory ("Enhanced Dynamic Random Access Memory", abbreviated as "EDRAM"), high bandwidth memory ("High Bandwidth Memory", abbreviated as "HBM"), hybrid memory cube ("Hybrid Memory Cube", abbreviated as "HMC"), ROM and RAM, etc.

[0100] Although the embodiments of the present disclosure are as described above, the contents are only examples used to facilitate understanding of the present disclosure, and are not intended to limit the scope and application scenarios of the present disclosure. Any technician in the technical field described in the present disclosure may make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present disclosure, but the scope of patent protection of the present disclosure shall still be based on the scope defined in the attached claims.

Claims

1. A method for testing a third-party device backend using a PyTorch test set, characterized in that: include: Load the pytest plugin for the PyTorch test suite; Through the pytest plug-in, the PyTorch test set is dynamically replaced using Python's monkey patch mechanism; Collect the replaced test cases; as well as The test case is run using the third-party device backend to perform testing.

2. The method according to claim 1, characterized in that The dynamic replacement of the PyTorch test set includes: Replace the PyTorch test set according to the specific backend settings of the third-party device backend; Replacing the PyTorch test set according to a custom interface of a third-party device backend; and The PyTorch test set is replaced according to the adaptation test case of the third-party device backend.

3. The method according to claim 2, characterized in that Replace the PyTorch test set with the specific backend settings of the third-party device backend including: Parse the backend information in the PyTorch interface input; and Replace some or all of the default settings in the backend information with the specific backend settings for the third-party device backend.

4. The method according to claim 3, characterized in that The backend information of parsing the PyTorch interface input parameter includes: Identify torch.device objects through Python decorators; and Extract the backend type information from the torch.device object.

5. The method according to claim 2, characterized in that: Replacing the PyTorch test set according to the custom interface of the third-party device backend includes: Identify custom interfaces to third-party device backends; and Replace the PyTorch interface with the custom interface that has the same or similar functionality as the custom interface.

6. The method according to claim 2, characterized in that Replacing the PyTorch test set according to the adaptation test case of the third-party device backend includes: Identify test cases that have functional limitations on third-party device backends; and The test cases are modified to adapt to the functional characteristics of the third-party device backend.

7. A device for testing the backend of a third-party device using a PyTorch test set, characterized in that: include: Loading module, used to load the pytest plugin for the PyTorch test suite; A replacement module, used to dynamically replace the PyTorch test set using the pytest plug-in and Python's monkey patch mechanism; A collection module, used for collecting replaced test cases; as well as The running module is used to run the test case using a third-party device backend to perform testing.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. A device for testing a third-party device backend using a PyTorch test set, characterized in that: include: processor; A memory for storing a computer program for testing a third-party device backend using a PyTorch test set; The processor is configured to execute the computer program to implement the method according to any one of claims 1 to 6.

10. A board for testing the backend of a third-party device using a PyTorch test set, characterized in that: Comprising the apparatus according to claim 7 or the device according to claim 9.

Images