Method, system, device and equipment for automatic analysis of calculation errors of AI accelerator cards

Abstract

The present application provides a method system, device and equipment for automatically analyzing calculation errors of an AI accelerator card, wherein the method includes: obtaining a first data stream and a second data stream, wherein the first data stream is a data stream between a first host and a first device in a first AI accelerator card when the two perform a calculation task, and the second data stream is a data stream between a second host and a second device in a second AI accelerator card when the two perform the calculation task; comparing data belonging to the same subtask in the first data stream and the second data stream to obtain an error, and obtaining the calculation errors of different AI accelerator cards by identifying data transmitted between the host and the device when the two perform a calculation task, and obtaining the same calculation result from the transmitted data for comparison, without changing the execution program of the calculation task or performing process analysis on the calculation task, and being able to quickly and easily obtain the error.

Description

Technical Field

[0001] The present application belongs to the field of integrated circuit technology, and specifically relates to a method, system, device and equipment for automatic analysis of calculation errors of an AI accelerator card. Background Art

[0002] In the related art, in order to troubleshoot the problem of inaccurate numerical calculation results during the calculation process of the AI ​​(Artificial Intelligence) accelerator card, the method usually used is: first, use a hardware and software environment that can obtain correct operating results to perform calculations. Then, using this as a reference, compare the calculation results of the calculation process with the AI ​​accelerator card under development step by step to troubleshoot. For example, a known AI accelerator card and its corresponding software environment will be selected to run the calculation task and obtain the calculation result of the AI ​​accelerator card. Then, use the AI ​​accelerator card under development and its corresponding software environment to run the same calculation task, obtain another calculation result, compare the two calculation results, and obtain the error.

[0003] In the process of troubleshooting the above calculation numerical errors, engineers are relied upon to modify the source code, or manually capture the calculation results for error analysis and comparison. This results in a low efficiency in the fault analysis process, difficulty in accurately locating the problem, and is prone to errors.

[0004] For example, to capture data, you need to change the numerical calculation program code according to the business process, manually capture data at the appropriate location or add data output code to capture data. This method requires analysis of the business process to clearly understand the location where data needs to be captured. If the business program source code is changed to output data, it will be invasive to the source code. In addition, a special program needs to be written to analyze the captured data. Summary of the invention

[0005] In view of this, the purpose of this application is to provide an AI accelerator card calculation error automatic analysis method, system, device and equipment to reduce the cost of error acquisition and speed up the acquisition speed.

[0006] In a first aspect, an embodiment of the present application provides an automatic analysis method for computing errors of an AI accelerator card, the method comprising: obtaining a first data stream and a second data stream, the first data stream being a data stream between a first host and a first device in a first AI accelerator card when the two perform a computing task, and the second data stream being a data stream between a second host and a second device in a second AI accelerator card when the two perform the computing task; comparing data belonging to the same subtask in the first data stream and the second data stream to obtain an error.

[0007] The automatic error analysis method of the embodiment of the present application identifies the data transmitted between the host and the device when the two perform a computing task, obtains the same calculation result from the transmitted data for comparison, and obtains the calculation error of different AI acceleration cards. There is no need to change the execution program of the computing task or perform process analysis on the computing task, and the error can be obtained quickly and easily.

[0008] In an optional embodiment of the present application, obtaining the first data stream includes: detecting a call to a transmission function, the transmission function being used to transmit data between the first host and the first device; when detecting that the transmission function is called, intercepting parameters transmitted to the transmission function; and capturing the data transmitted between the first host and the first device based on the parameters to obtain the first data stream.

[0009] In the above embodiment, by detecting the calling status of the transmission function, the transmission data between the host and the device can be obtained without changing the computing program.

[0010] In an optional embodiment of the present application, when the transmission function is a synchronous transmission function, the data transmitted between the first host and the first device is captured based on the parameters, including: obtaining the data transmission stage between the first host and the first device; if the data transmission stage is the stage in which the first host transmits data to the first device, determining the starting address and the memory size based on the parameters, and obtaining the first memory data in the memory of the first host based on the starting address and the memory size; if the data transmission stage is the stage in which the first device transmits data to the first host, calling the transmission function to store the data transmitted by the first device to the first host in the memory of the first host; determining the starting address and the memory size based on the parameters, and obtaining the second memory data in the memory of the first host based on the starting address and the memory size; wherein the first data stream includes: first memory data and / or the second memory data.

[0011] In the above embodiment, the transmission data between the host and the device is obtained through the transmission function, thereby obtaining the data of the H2D, computing core and D2H sequences, and the input and output data of the specified position can be obtained through the sequence data.

[0012] In an optional embodiment of the present application, when the transmission function is an asynchronous transmission function, capturing the data transmitted between the first host and the first device based on the parameters includes: obtaining the data transmission stage between the first host and the first device; if the data transmission stage is the stage in which the first host transmits data to the first device, determining the starting address and the memory size based on the parameters; obtaining memory data in the memory of the first host based on the starting address and the memory size; if the data transmission stage is the stage in which the first device transmits data to the first host, calling the transmission function to store the data transmitted by the first device to the first host in the memory of the first host; after waiting for all operations in the operation flow currently executed by the first host to be executed, determining the starting address and the memory size based on the parameters, and obtaining memory data in the memory of the first host based on the starting address and the memory size.

[0013] In the above embodiment, the transmission data between the host and the device is obtained through the transmission function, thereby obtaining the data of the H2D, computing core and D2H sequences, and the input and output data of the specified position can be obtained through the sequence data.

[0014] In an optional embodiment of the present application, capturing the data transmitted between the first host and the first device based on the parameters includes: determining attribute information of the data transmitted between the first host and the first device based on the parameters, the attribute information including: the stage of data transmission and / or data attributes; judging whether the attribute information meets the data capture conditions; if the attribute information meets the data capture conditions, capturing the data transmitted between the first host and the second device based on the parameters.

[0015] In the above embodiment, the data capture conditions are set so that data can be captured in a targeted manner, thereby enabling rapid and accurate data comparison.

[0016] In an optional embodiment of the present application, the data capture condition includes at least one of the following: the stage of data transmission is the stage of data transmission from the first device to the first host, the stage of data transmission is the stage of data transmission from the first host to the first device, the data attribute is a specified serial number, and the data attribute is a specified data type.

[0017] In a second aspect, an embodiment of the present application provides an automatic analysis system for computing errors of an AI accelerator card, the automatic analysis system for computing errors of an AI accelerator card comprising: an analysis device, a first AI accelerator card and a second AI accelerator card; the first AI accelerator card comprises a first host and a first device, and the first AI accelerator card is used to perform a computing task; the second AI accelerator card comprises a second host and a second device, and the second AI accelerator card is used to perform a computing task; the analysis device is used to obtain a first data stream and a second data stream, the first data stream being the data stream between the first host and the first device in the first AI accelerator card when the two perform a computing task, and the second data stream being the data stream between the second host and the second device in the second AI accelerator card when the two perform the computing task; and comparing data belonging to the same subtask in the first data stream and the second data stream to obtain an error.

[0018] In a third aspect, an embodiment of the present application provides an automatic analysis device for computing errors of an AI acceleration card, and the automatic analysis device for computing errors of an AI acceleration card includes: an acquisition module, used to acquire a first data stream and a second data stream, the first data stream being a data stream between a first host and a first device in a first AI acceleration card when the two perform a computing task, and the second data stream being a data stream between a second host and a second device in a second AI acceleration card when the two perform the computing task; a comparison module, used to compare data belonging to the same subtask in the first data stream and the second data stream to obtain an error.

[0019] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: a memory and a processor, the processor being connected to the memory; the memory being used to store programs; and the processor being used to call the programs stored in the memory to execute any of the methods described in the first aspect.

[0020] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method described in any one of the first aspects is executed.

[0021] Other features and advantages of the present application will be described in the following description. The purpose and other advantages of the present application can be realized and obtained through the structures specifically pointed out in the written description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. The above and other purposes, features and advantages of the present application will be more clear as shown in the drawings.

[0023] Figure 1 A schematic diagram showing data transmission in an AI acceleration card in the prior art is shown;

[0024] Figure 2 A schematic diagram of a process flow of an automatic analysis method for calculation errors of an AI accelerator card provided in an embodiment of the present application is shown;

[0025] Figure 3 A schematic block diagram of an AI accelerator card calculation error automatic analysis system provided in an embodiment of the present application is shown;

[0026] Figure 4 A schematic block diagram of an automatic analysis device for calculation errors of an AI acceleration card provided in an embodiment of the present application is shown;

[0027] Figure 5 A schematic block diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0028] Hereinafter, embodiments of the present application are described in detail with reference to the accompanying drawings. It should be noted that although the same elements are shown in different drawings, they will be represented by the same reference numerals. In the following description, specific details such as detailed configuration and components are provided to only help fully understand the embodiments of the present application. Therefore, it should be clear to those skilled in the art that various changes and modifications to the embodiments described herein can be made without departing from the scope of the present application. In addition, for clarity and conciseness, the description of well-known functions and configurations is omitted. The terms described below are defined in consideration of the functions in the present application, and may be different according to the user, the user's intention or custom. Therefore, the definition of the term should be determined based on the content throughout this specification.

[0029] The present application may have various modifications and various embodiments, in which the embodiments are described in detail below with reference to the accompanying drawings. However, it should be understood that the present application is not limited to the embodiments, but includes all modifications, equivalents and substitutes within the scope of the present application.

[0030] The terms used herein are only used to describe various embodiments of the present application and are not intended to limit the present application. Unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In the present application, it should be understood that the term "including" or "having" indicates the presence of features, quantities, steps, operations, structural elements, parts or combinations thereof, and does not exclude the presence of one or more other features, quantities, steps, operations, structural elements, parts or combinations thereof, or the possibility of adding one or more other features, quantities, steps, operations, structural elements, parts or combinations thereof.

[0031] Unless defined differently, all terms used herein have the same meaning as understood by a person skilled in the art to which the present application belongs. Unless explicitly defined in the present application, terms (such as those defined in a general dictionary) should be interpreted as having the same meaning as the contextual meaning in the relevant art, and should not be interpreted as having an idealized or overly formal meaning.

[0032] The electronic device according to one embodiment may be one of various types of electronic devices. The electronic device may include, for example, a portable communication device (e.g., a smart phone), a computer, a portable multimedia device, a portable medical device, a camera, a wearable device, or a household appliance. According to one embodiment disclosed, the electronic device is not limited to those described above.

[0033] The terms used in this application are not intended to limit the application, but are intended to include various changes, equivalents or alternatives of corresponding embodiments. Regarding the description of the accompanying drawings, similar reference numerals may be used to represent similar elements or related elements. Unless the relevant context clearly indicates otherwise, the singular form of the noun corresponding to the item may include one or more things. As used herein, each of the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C" and "at least one of A, B or C" may include all possible combinations of the items listed together in the corresponding one in the phrase. As used herein, terms such as "the 1st", "the 2nd", "the first" and "the second" may be used to distinguish the corresponding component from another component, and are not intended to limit the component in other aspects (e.g., importance or order). It is intended that if an element (e.g., a first element) is referred to as being “coupled with,” “coupled to,” “connected to,” or “connected to” another element (e.g., a second element), with or without the term “operably” or “communicatively,” it indicates that the element may be coupled with the other element directly (e.g., wired), wirelessly, or via a third element.

[0034] As used herein, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with other terms (e.g., "logic," "logic block," "component," and "circuit"). A module may be a single integrated component or its smallest unit or component suitable for performing one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application specific integrated circuit (ASIC).

[0035] To facilitate understanding, the relevant terms and concepts involved in the embodiments of the present application are first introduced below.

[0036] The AI ​​accelerator card is a heterogeneous computing architecture. The heterogeneous computing programming models that support the AI ​​accelerator card include CUDA (Compute Unified Device Architecture) and OpenCL (Open Computing Language).

[0037] The programming model of AI accelerator cards includes three stages: H2D (Host to Device), kernel computation, and D2H (Device to Host). In the H2D stage, data is transferred from the host memory to the memory of the device (such as a graphics processor GPU). Then in the kernel computation stage, the device performs specific computing tasks based on the received data and processes the data in parallel. Finally, in the D2H stage, the result data is transferred from the device memory back to the host memory for subsequent processing. These three stages together constitute the complete process of heterogeneous computing of AI accelerator cards, giving full play to the advantages of AI accelerator cards in data parallel processing and accelerating the completion of computing tasks. For example, for the CUDA programming model, the CUDA programming model mainly includes the following key components: Host: refers to the CPU and its memory, which is responsible for managing data transmission and controlling the kernel; Device: refers to the GPU and its memory, which is used to perform parallel computing tasks; CUDA kernel: a parallel function executed on the GPU, used to process large amounts of data.

[0038] The programming model also includes runtime, which is the core part of programming models such as CUDA and OpenCL. Through core functions such as device management, memory management, and thread management, it allows developers to use AI accelerator cards for high-performance computing to simplify the implementation and management of parallel computing.

[0039] In the CUDA programming model, the process of data transfer and kernel execution is as follows: H2D data transfer: The data to be processed is transferred from the CPU memory (Host) to the GPU memory (Device), which is called H2D data transfer. CUDA kernel execution: After the data transfer is completed, the GPU executes the CUDA kernel for parallel computing. The CUDA kernel is a function that is executed in parallel on the GPU and can process a large amount of data at the same time. In the CUDA kernel, threads are organized into thread blocks, and multiple thread blocks can be executed simultaneously. Threads within a thread block can communicate through shared memory, and threads between different thread blocks can communicate through global memory. D2H data transfer: The calculation results are transferred from the GPU memory back to the CPU memory, which is called D2H data transfer.

[0040] like Figure 1 As shown, Figure 1 A schematic diagram of data transmission in an AI accelerator card provided by an embodiment of the present application is shown. Part of the calculation process is executed on the Host (host) such as the CPU, and the program that executes this part is called the Host program (host program), and the other part of the calculation process is executed on the device such as the graphics processor GPU. The program of this part of the calculation process is called the Device program (device program) or the GPU program (graphics processor program). The Host program is responsible for initializing the environment, including the AI ​​accelerator card operating environment and parameters. The Host program then copies the input data from the CPU Memory (central processing unit memory) to the device's video memory such as the DRAM (dynamic random access memory) in the GPU. This data transmission 110 is a H2D data transmission stage. The device caches the received data and loads the Device program for calculation to obtain the calculation result. The Host program finally copies the calculation result from the video memory to the CPU Memory. This data transmission 120 is a D2H data transmission stage.

[0041] like Figure 2 As shown, Figure 2 FIG. 1 is a flow chart of an automatic analysis method for AI acceleration card calculation errors provided in an embodiment of the present application. Figure 2 As shown, the method includes steps 210 to 240:

[0042] Step 210: Obtain a first data stream and a second data stream, wherein the first data stream is a data stream between a first host and a first device in a first AI accelerator card when the two perform a computing task, and the second data stream is a data stream between a second host and a second device in a second AI accelerator card when the two perform the computing task.

[0043] A first data stream is obtained, where the first data stream is a data stream between a first host and a first device when the two devices cooperate with each other to perform a preset computing task. A second data stream is obtained, where the second data stream is a data stream between a second host and a second device when the two devices cooperate with each other to perform a preset computing task.

[0044] For example, the first host is a first central processing unit, and the first device is a first image processor, and the two constitute a first AI accelerator card. In the first AI accelerator card, a first programming model such as a CUDA programming model is used to perform a first computing task. The second host is a second central processing unit, and the first device is a second image processor, and the two constitute a second AI accelerator card. The second programming model is used to perform the first computing task in the second AI accelerator card. One of the second AI accelerator card and the first AI accelerator card is a reference AI accelerator card, and the other is an AI accelerator card to be tested. The host, device, and programming environment in the reference AI accelerator card are all verified and can perform computing tasks correctly. At least one of the host, device, and programming environment in the AI ​​accelerator card to be tested is unverified, and errors may occur during numerical calculations.

[0045] For example, the first AI accelerator card is used to perform the first computing task, and the first data stream between the first host and the first device in the AI ​​accelerator card is obtained, and the first data stream is used as a benchmark. The second AI accelerator card is used to perform the first computing task. Due to the unverified hardware or software in the second AI accelerator card, computing errors may occur for various reasons. At this time, referring to the calculation results of the first AI accelerator card, modifying the source code, and comparing the calculation results in the calculation process step by step can eliminate the fault.

[0046] In some embodiments of the present application, the first host and the second host may be the same host. In this architecture, the first host and the first device first form a first AI accelerator card, and the first AI accelerator card performs a computing task. While performing the computing task, the first host obtains a first data stream between the first host and the first device. After the first AI accelerator card completes the computing task, the first host disconnects from the first device, and the first host establishes a connection with the second device to form a second AI accelerator card. The second AI accelerator card performs a computing task. While performing the computing task, the first host obtains a second data stream between the first host and the second device.

[0047] In one embodiment of the present application, obtaining the first data stream includes: detecting the call of a transmission function, the transmission function being used to transmit data between the first host and the first device; when detecting that the transmission function is called, intercepting parameters transmitted to the transmission function; and capturing the data transmitted between the first host and the first device based on the parameters to obtain the first data stream.

[0048] In a heterogeneous AI accelerator card, the H2D stage transfers the data to be processed from the CPU memory to the AI ​​accelerator card memory, and the D2H stage transfers the calculation results from the AI ​​accelerator card memory back to the CPU memory.

[0049] For example, an external analysis device is used to collect the first data stream between the first host and the first device and the second data stream between the second host and the second device. The analysis device can obtain the transmission data by intercepting the transmission function used in the H2D stage and the D2H stage. For example, the analysis device includes a hook interface unit, which is used to intercept and expand the function of the runtime and provide various information capture capabilities. The hook interface unit obtains various information capture capabilities by managing preset hook functions. For the setting of the hook function, the CUDA runtime library can be modified to insert the hook function code in the data transmission function. Specifically, the hook function code is added at the entry and exit of the data transmission function to intercept the data transmission event. The main function of the hook function code is to obtain relevant information of the data transmission, such as the size of the transmitted data, the transmission direction (D2H or H2D), etc. For example, a dynamic link library (DLL) is developed in advance, which contains a custom hook function and API hook logic. By modifying the import address table (IAT) of runtime libraries such as CUDA or OpenCL or using dynamic link library (DLL) hijacking technology, the original API function address is replaced with a custom hook function address. In the hook function, the original API function is first called to maintain the original functionality. Then, the data transfer monitoring and modification logic is added as needed. For example, the CUDADriver API or OpenCL API can be used to obtain the current data transfer event and record the relevant information.

[0050] When the AI ​​accelerator card application is running, API calls are intercepted dynamically. For example, the runtime API Hook intercepts CUDA or OpenCL data transfer functions to obtain data transfer information in the D2H and H2D stages.

[0051] Preferably, the data transmitted by the hook function is monitored in real time and numerically analyzed. For example, a monitoring and analysis module is provided to be responsible for receiving the data transmission information transmitted by the hook function, and to perform real-time monitoring and numerical analysis. The monitoring and analysis module can verify the transmitted data to ensure the integrity and accuracy of the data. In addition, the monitoring and analysis module can also collect information such as the number of times and size of statistical transmissions in order to analyze the algorithm performance and optimize the data transmission strategy. Further, the monitoring and analysis module displays the analysis results to the user and provides optimization suggestions. For example, for bottlenecks in the data transmission process, the monitoring and analysis module can suggest that the user optimize the data transmission strategy, such as reducing the number of data transmissions, increasing the amount of data transmitted in a single time, etc. In addition, the monitoring and analysis module can also provide the user with suggestions for optimizing the algorithm performance based on the statistical information of data transmission.

[0052] During the operation of the programming model, the hook function will detect the call of the transfer function by the internal program of the first host, and the transfer function is used to transfer data between the first host and the first device. For example, CUDA provides two types of APIs, cudaMemcpy or cudaMemcpyAsync, for data transfer between the host and the device.

[0053] When the hook function detects that the transmission function is called, it will intercept the parameters transmitted to the transmission function. Further, according to these parameters, the data transmitted between the first host and the first device is captured to obtain the first data stream. The Hook technology is used to intercept the transmission function call and data transmission in the programming model runtime library, so as to obtain the data transmission information in the D2H and H2D stages.

[0054] In the above embodiment, by detecting the calling status of the transmission function, the transmission data between the host and the device can be obtained without changing the computing program.

[0055] In one embodiment of the present application, when the transmission function is a synchronous transmission function, the data transmitted between the first host and the first device is captured based on the parameters, including: obtaining the data transmission stage between the first host and the first device; if the data transmission stage is the stage in which the first host transmits data to the first device, determining the starting address and the memory size based on the parameters, and obtaining the first memory data in the memory of the first host based on the starting address and the memory size; if the data transmission stage is the stage in which the first device transmits data to the first host, calling the transmission function to store the data transmitted by the first device to the first host in the memory of the first host; determining the starting address and the memory size based on the parameters, and obtaining the second memory data in the memory of the first host based on the starting address and the memory size; wherein the first data stream includes: the first memory data and / or the second memory data.

[0056] The data transmission between the host and the device can be divided into the H2D and D2H stages. Since the data transmission directions are different in different data transmission stages, the data processing processes are also different. Therefore, different APIs are provided for the H2D and D2H stages to complete the transmission between the host memory and the device memory. In addition, in some implementation scenarios, in order to meet different transmission needs, multiple transmission functions are set for calling. For example, synchronous transmission functions and asynchronous transmission functions are set.

[0057] In the case of detecting the calling of the synchronous transmission function, when it is detected that the synchronous transmission function is called, the parameters transmitted to the synchronous transmission function are intercepted, and the stage of calling the synchronous transmission function for data transmission is determined.

[0058] If the data transmission stage is H2D, for example, the data transmission stage is a stage in which the first host transmits data to the first device. Then, based on the intercepted parameters, the starting address and memory size of the transmitted data in the host memory are determined. After the starting address and memory size are calculated, the data to be transmitted are obtained based on this information, and the obtained data is the first memory data.

[0059] After the data is acquired, the default API function is executed. The default API refers to the API whose parameters are intercepted, which is the synchronous transmission function in this embodiment. Executing the API function is to realize the function executed by the API when the API is called. There are various methods for realizing the default API function in this application. For example, the intercepted parameters are retransmitted to the API of the synchronous transmission function to realize the purpose of executing the default API function. Alternatively, codes can be added in the hook function, which are used to transmit data after the data is acquired.

[0060] If the data transmission phase is D2H, such as the data transmission phase is the phase of the first device transmitting data to the first host. The default API function is first executed, such as retransmitting the intercepted parameters to the API of the synchronous transmission function for data transmission. Then, based on the intercepted parameters, the starting address and memory size of the transmitted data in the host memory are determined. After calculating the starting address and memory size, the data to be transmitted are obtained based on this information, and the obtained data is the second memory data.

[0061] Preferably, when acquiring memory data, meta information such as the IP address of the running machine, the name of the execution file of the application task, and the process number can also be acquired. These meta information identify the attributes of the first memory data and can be used to distinguish data or analyze. The above meta information and memory data are transmitted to the analysis platform of the analysis device via the network.

[0062] The first data stream includes at least one of the first memory data and the second memory data. For example, the call detection of the transmission function is performed during all the time when the first host executes the computing task. Since the computing task includes the H2D stage and the D2H stage, two types of memory data will be obtained. For another example, the call of the transmission function can be detected when only a subtask in the computing task is performed, and the memory data is obtained according to the parameters when it is called. If the data transmission stage in the subtask is the H2D stage, the first data stream only includes the first memory data.

[0063] In the above embodiment, the transmission data between the host and the device is obtained through the transmission function, thereby obtaining the data of the H2D, computing core and D2H sequences, and the input and output data of the specified position can be obtained through the sequence data.

[0064] In one embodiment of the present application, when the transmission function is an asynchronous transmission function, capturing the data transmitted between the first host and the first device based on the parameters includes: obtaining the data transmission stage between the first host and the first device; if the data transmission stage is the stage in which the first host transmits data to the first device, determining the starting address and the memory size based on the parameters; obtaining memory data in the memory of the first host based on the starting address and the memory size; if the data transmission stage is the stage in which the first device transmits data to the first host, calling the transmission function to store the data transmitted by the first device to the first host in the memory of the first host; after waiting for all operations in the operation flow currently executed by the first host to be executed, determining the starting address and the memory size based on the parameters, and obtaining memory data in the memory of the first host based on the starting address and the memory size.

[0065] The asynchronous transfer function allows an asynchronous data transfer operation to be initiated, which means that the host will not block and wait after calling the function, but will return immediately and continue to execute subsequent instructions. For example, cudaMemcpyAsync is an asynchronous function in the CUDA programming model.

[0066] For example, an external analysis device is used to detect the call of an asynchronous transmission function, and the analysis device includes: a hook interface unit, a data stream acquisition unit and a data capture unit. The data stream acquisition unit is used to obtain transmission data and H2D, calculation core, D2H operation sequences in the transmission data through the hook interface unit, and identify loop subsequences. The data capture unit is used to capture calculation data according to configuration data and store it in a file.

[0067] When the analysis device detects that the asynchronous transmission function is called, it intercepts the parameters transmitted to the asynchronous transmission function and determines the stage of calling the asynchronous transmission function for data transmission.

[0068] If the data transmission stage is H2D, for example, the data transmission stage is a stage in which the first host transmits data to the first device. Then, based on the intercepted parameters, the starting address and memory size of the transmitted data in the host memory are determined. After the starting address and memory size are calculated, the data to be transmitted are obtained based on these information, and the obtained data is the first memory data. After the data is obtained, the default API function is executed.

[0069] If the data transmission phase is D2H, such as the data transmission phase is a phase in which the first device transmits data to the first host, the default API function is first executed, such as retransmitting the intercepted parameters to the API of the synchronous transmission function for data transmission.

[0070] Subsequently, after all operations in the operation stream currently executed by the first host are completed, the starting address and memory size of the data to be transmitted in the host memory are determined according to the intercepted parameters. After the starting address and memory size are calculated, the data to be transmitted are obtained according to this information, and the obtained data is the second memory data. In the programming model of the AI ​​accelerator card, streams are usually used to organize tasks. The operation stream refers to a series of operations that the first host is currently set to perform. For example, in the CUDA programming model, the CUDA stream is used to organize and manage the tasks to be performed. The computing task usually refers to a series of related operations performed on the AI ​​accelerator card, which can be decomposed into multiple subtasks or work units. Each work unit can be executed by associating with different CUDA streams. For example, a callback function is pre-registered, and the callback function will only run on the host when all previously arranged operations in the stream are correct. The callback function is used to determine the starting address and memory size of the host memory based on the parameters, and after the starting address and memory size are calculated, the data to be transmitted is obtained based on this information. The callback function can also be used to obtain metadata such as the IP address of the running machine, the execution file name of the application task, and the process number.

[0071] After collecting the calculation results, the analysis device will store the results in a database or upload the results to an analysis platform in the analysis device.

[0072] In the above embodiment, the transmission data between the host and the device is obtained through the transmission function, thereby obtaining the data of the H2D, computing core and D2H sequences, and the input and output data of the specified position can be obtained through the sequence data.

[0073] In one embodiment of the present application, capturing the data transmitted between the first host and the first device based on the parameters includes: determining attribute information of the data transmitted between the first host and the first device based on the parameters, the attribute information including: the stage of data transmission and / or data attributes; judging whether the attribute information meets the data capture conditions; if the attribute information meets the data capture conditions, capturing the data transmitted between the first host and the second device based on the parameters.

[0074] Before data acquisition, data capture conditions can be set. For example, a configuration management unit can be set on the analysis device, and the configuration management unit is used to display information about transmitted data and provide users with configurations on how to capture data. Data capture conditions include data transmission stages and / or data attributes.

[0075] Therefore, before the data to be transmitted is captured, the attribute information of the data to be transmitted is obtained, and the attribute information includes: the data transmission stage and / or data attribute, the data transmission stage includes the H2D stage, the D2H stage, etc., and the data attribute includes: data types such as int8, in16, int32, in64, float, double, FP and other basic data types, and can also be complex data types composed of basic data types. Data attributes can also include the sequence number of the data transmission stage, the starting position of the data in the memory, etc. The data attributes correspond to the data capture conditions.

[0076] Determine whether the attribute information meets the data capture condition. If the attribute information meets the data capture condition, capture the data transmitted between the first host and the second device according to the parameter. The captured data is stored in a specified location in a unified format in an asynchronous manner.

[0077] In one embodiment of the present application, the data capture conditions include at least one of the following: the stage of data transmission is the stage of data transmission from the first device to the first host, the stage of data transmission is the stage of data transmission from the first host to the first device, the data attribute is a specified serial number, and the data attribute is a specified data type.

[0078] For example, you can set the data transmission stage for grabbing data, specify the H2D and D2H operations for grabbing data. Another example is that it supports multiple grabbing methods, such as grabbing the starting position and length, conditional grabbing (grabbing only when the conditions are met), and grabbing data with a specified sequence number or sequence number range.

[0079] In the above embodiment, the data capture conditions are set so that data can be captured in a targeted manner, thereby enabling rapid and accurate data comparison.

[0080] Step 210: Compare the data belonging to the same subtask in the first data stream and the second data stream to obtain an error.

[0081] The numerical calculation process of heterogeneous AI accelerator cards is completed by the cooperation of many calculation processes. Each calculation process is completed by various combinations of the three basic operations of H2D, calculation core and D2H, and the calculation error (error) is usually generated by the device in the calculation core stage. For the same numerical calculation, the calculation process is the same, so the H2D and D2H data of each calculation process are compared for evaluation. The calculation error of one type of AI accelerator card can be evaluated relative to another type of AI accelerator card.

[0082] Determine the correspondence between the reference AI accelerator card result and the data collection sequence of the AI ​​accelerator card task to be analyzed. For example, after obtaining the first data stream and the second data stream, the data transmission stages therein are matched one by one.

[0083] If a pair is matched, the numerical error analysis of the calculation results is started and the results of the error analysis are presented. Based on the configuration, the application can be terminated in the case of serious and unreasonable errors to avoid meaningless calculations and consumption of computing power.

[0084] In an example of the present application, in an architecture where the first host and the second host are the same host, after the first host obtains the second data stream, the data belonging to the same subtask in the first data stream and the second data stream are compared one by one, for example, two data belonging to the same data transmission stage in the two data streams are obtained and compared to obtain the error.

[0085] In another example of the present application, the analysis device further includes: a calculation error configuration management unit and a calculation error analysis unit, wherein the calculation error configuration management unit is used to configure the error analysis mode according to the transmission data information. The calculation error analysis unit is used to analyze the error according to the transmission data information and the error analysis configuration data, and output an error analysis report.

[0086] The calculation error analysis unit can analyze errors in a variety of ways, such as: point-to-point comparison of absolute error, relative error, precision comparison (comparison to the number of decimal places or integer places), and counting the number of data that reach a certain coding error. Furthermore, the calculation error analysis unit supports dynamic insertion of comparison functions for complex data types.

[0087] The automatic analysis method for AI acceleration card calculation errors in the embodiment of the present application identifies the data transmitted between the host and the device when the two perform a calculation task, obtains the same calculation result from the transmitted data for comparison, and obtains the calculation errors of different AI acceleration cards. There is no need to change the execution program of the calculation task or perform process analysis on the calculation task, and the error can be obtained quickly and easily.

[0088] The embodiment of the present application can automatically identify the transmission data between the host and the device in the business program, as well as all H2D, computing core and D2H sequences in each data transmission stage, and then capture the data transmitted by the specified H2D and D2H in the specified data transmission stage according to the set conditions, and display the calculation error through a general calculation error analysis program. By capturing the D2H and H2D stages, the error process of locating the calculation result is automated, thereby improving the fault analysis and diagnosis process of numerical error calculation of the AI ​​accelerator card.

[0089] like Figure 3 As shown, Figure 3 An AI acceleration card calculation error automatic analysis system 300 provided in an embodiment of the present application is shown, wherein the system 300 includes an analysis device 310, a first AI acceleration card 320, and a second AI acceleration card 330;

[0090] The first AI acceleration card 320 includes a first host 321 and a first device 322, and the first AI acceleration card is used to perform computing tasks;

[0091] The second AI acceleration card 330 includes a second host 331 and a second device 332, and the second AI acceleration card is used to perform computing tasks;

[0092] The analysis device 310 is used to obtain a first data stream and a second data stream, where the first data stream is a data stream between a first host and a first device in the first AI accelerator card when the two perform a computing task, and the second data stream is a data stream between a second host and a second device in the second AI accelerator card when the two perform the computing task; compare data belonging to the same subtask in the first data stream and the second data stream to obtain an error.

[0093] In the embodiment of the AI ​​accelerator card calculation error automatic analysis system 300, the unintroduced parts can refer to the corresponding contents in the aforementioned method embodiment. Its implementation principle and technical effects are the same as those in the aforementioned method embodiment. For the sake of brief description, the embodiment of the system 300 will not be repeated.

[0094] like Figure 4 As shown, Figure 4 An AI accelerator card calculation error automatic analysis device 400 provided in an embodiment of the present application is shown. The AI ​​accelerator card calculation error automatic analysis device 400 includes: an acquisition module 410 and a comparison module 420;

[0095] An acquisition module 410 is configured to acquire a first data stream and a second data stream, wherein the first data stream is a data stream between a first host and a first device in a first AI accelerator card when the two devices perform a computing task, and the second data stream is a data stream between a second host and a second device in a second AI accelerator card when the two devices perform the computing task;

[0096] The comparison module 420 is used to compare the data belonging to the same subtask in the first data stream and the second data stream to obtain an error.

[0097] In one embodiment of the present application, the acquisition module 410 includes:

[0098] a detection module, used to detect the calling of a transmission function, where the transmission function is used to transmit data between the first host and the first device;

[0099] The capture module is used to intercept the parameters transmitted to the transmission function when detecting that the transmission function is called; capture the data transmitted between the first host and the first device according to the parameters to obtain the first data stream.

[0100] In one embodiment of the present application, when the transmission function is a synchronous transmission function, the capture module is specifically used to obtain the data transmission stage between the first host and the first device; if the data transmission stage is the stage in which the first host transmits data to the first device, the starting address and the memory size are determined according to the parameters, and the first memory data is obtained in the memory of the first host according to the starting address and the memory size; if the data transmission stage is the stage in which the first device transmits data to the first host, the transmission function is called to store the data transmitted by the first device to the first host in the memory of the first host; the starting address and the memory size are determined according to the parameters, and the second memory data is obtained in the memory of the first host according to the starting address and the memory size; wherein the first data stream includes: first memory data and / or the second memory data.

[0101] In one embodiment of the present application, when the transmission function is a synchronous transmission function, the capture module is specifically used to obtain the data transmission stage between the first host and the first device; if the data transmission stage is the stage in which the first host transmits data to the first device, the starting address and the memory size are determined according to the parameters; the memory data is obtained in the memory of the first host according to the starting address and the memory size; if the data transmission stage is the stage in which the first device transmits data to the first host, the transmission function is called to store the data transmitted by the first device to the first host in the memory of the first host; after waiting for all operations in the operation flow currently executed by the first host to be executed, the starting address and the memory size are determined according to the parameters, and the memory data is obtained in the memory of the first host according to the starting address and the memory size.

[0102] In one embodiment of the present application, the acquisition module 410 is specifically used to determine the attribute information of the data transmitted between the first host and the first device based on the parameters, and the attribute information includes: the stage of data transmission and / or data attributes; determine whether the attribute information meets the data capture conditions; if the attribute information meets the data capture conditions, capture the data transmitted between the first host and the second device based on the parameters.

[0103] In one embodiment of the present application, the data capture condition includes at least one of the following:

[0104] The data transmission stage is the stage in which the first device transmits data to the first host, the data transmission stage is the stage in which the first host transmits data to the first device, the data attribute is a specified sequence number, and the data attribute is a specified data type.

[0105] The AI ​​acceleration card calculation error automatic analysis device provided in the embodiment of the present application has the same implementation principle and technical effects as those in the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0106] like Figure 5 As shown, Figure 5 A schematic block diagram of an electronic device 500 provided in an embodiment of the present application is shown. The electronic device 500 includes: a processor 510 and a memory 520;

[0107] The processor 510, the memory 520 and each component are directly or indirectly electrically connected to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory 520 is used to store computer programs, such as storing Figure 4 The software function module shown in is the AI ​​accelerator card calculation error automatic analysis device 400. Among them, the AI ​​accelerator card calculation error automatic analysis device 400 includes at least one software function module that can be stored in the memory 520 in the form of software or firmware (Firmware) or solidified in the operating system (OS) of the electronic device 500. The processor 510 is used to execute the executable module stored in the memory 520, such as the software function module or computer program included in the AI ​​accelerator card calculation error automatic analysis device 400.

[0108] Among them, the memory 520 can be, but is not limited to, random access memory (Random Access Memory, RAM), read only memory (Read Only Memory, ROM), programmable read-only memory (Programmable Read-Only Memory, PROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, EPROM), electrically erasable read-only memory (Electric Erasable Programmable Read-Only Memory, EEPROM), etc.

[0109] The processor 510 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a microprocessor, etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The various methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. Alternatively, the processor 510 may also be any conventional processor, etc.

[0110] An embodiment of the present application also provides a non-volatile computer-readable storage medium (hereinafter referred to as storage medium), on which a computer program is stored. When the computer program is run by a computer such as the above-mentioned electronic device 500, the above-mentioned AI acceleration card calculation error automatic analysis method is executed.

[0111] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0112] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0113] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0114] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a computer-readable storage medium, including several instructions for enabling a computer device (which can be a personal computer, a laptop, a server, or an electronic device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned computer-readable storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0115] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An automatic analysis method for AI accelerator card calculation errors, characterized in that: The method comprises: Obtain a first data stream and a second data stream, where the first data stream is a data stream between a first host and a first device in a first AI accelerator card when the two devices perform a computing task, and the second data stream is a data stream between a second host and a second device in a second AI accelerator card when the two devices perform the computing task; Comparing data belonging to the same subtask in the first data stream and the second data stream to obtain an error; The obtaining of the first data stream includes: detecting a call of a transmission function, the transmission function being used to transmit data between the first host and the first device; When detecting that the transfer function is called, intercepting the parameters transmitted to the transfer function; capturing data transmitted between the first host and the first device according to the parameter to obtain the first data stream; The method of capturing data transmitted between the first host and the first device according to the parameter to obtain the first data stream includes: Obtain the data transmission phase between the first host and the first device; if the data transmission phase is a phase in which the first host transmits data to the first device, determine the starting address and the memory size according to the parameters, and obtain memory data in the memory of the first host according to the starting address and the memory size.

2. The method according to claim 1, characterized in that When the transmission function is a synchronous transmission function, capturing the data transmitted between the first host and the first device according to the parameter further includes: If the data transmission phase is a phase in which the first device transmits data to the first host, the transmission function is called to store the data transmitted by the first device to the first host in the memory of the first host; a starting address and a memory size are determined according to the parameters, and second memory data are obtained in the memory of the first host according to the starting address and the memory size; The first data stream includes: first memory data and the second memory data, wherein the first memory data is memory data obtained in the memory of the first host according to the starting address and the memory size during the stage of transmitting data from the first host to the first device.

3. The method according to claim 1, characterized in that When the transmission function is an asynchronous transmission function, capturing the data transmitted between the first host and the first device according to the parameter further includes: If the data transmission phase is the phase in which the first device transmits data to the first host, the transmission function is called to store the data transmitted by the first device to the first host into the memory of the first host; after waiting for all operations in the operation flow currently executed by the first host to be executed, the starting address and the memory size are determined according to the parameters, and the memory data is obtained in the memory of the first host according to the starting address and the memory size.

4. The method according to claim 1, characterized in that The capturing of data transmitted between the first host and the first device according to the parameter includes: Determining attribute information of data transmission between the first host and the first device according to the parameter, the attribute information including: a data transmission stage and / or a data attribute; Determining whether the attribute information meets the data capture condition; If the attribute information satisfies the data capture condition, the data transmitted between the first host and the second device is captured according to the parameter.

5. The method according to claim 4, characterized in that The data capture conditions include at least one of the following: The data transmission stage is the stage in which the first device transmits data to the first host, the data transmission stage is the stage in which the first host transmits data to the first device, the data attribute is a specified sequence number, and the data attribute is a specified data type.

6. An AI accelerator card calculation error automatic analysis system, characterized in that: The AI ​​accelerator card calculation error automatic analysis system includes an analysis device, a first AI accelerator card and a second AI accelerator card; The first AI accelerator card includes a first host and a first device, and the first AI accelerator card is used to perform computing tasks; The second AI accelerator card includes a second host and a second device, and the second AI accelerator card is used to perform computing tasks; The analysis device is used to obtain a first data stream and a second data stream, where the first data stream is a data stream between a first host and a first device in the first AI accelerator card when the two perform a computing task, and the second data stream is a data stream between a second host and a second device in the second AI accelerator card when the two perform the computing task; compare data belonging to the same subtask in the first data stream and the second data stream to obtain an error; The analysis device is used to: detect the call of a transmission function, the transmission function is used to transmit data between the first host and the first device; when detecting that the transmission function is called, intercept the parameters transmitted to the transmission function; and capture the data transmitted between the first host and the first device according to the parameters to obtain the first data stream; Among them, capturing the data transmitted between the first host and the first device according to the parameters to obtain the first data stream includes: obtaining the data transmission stage between the first host and the first device; if the data transmission stage is a stage in which the first host transmits data to the first device, determining the starting address and the memory size according to the parameters, and obtaining memory data in the memory of the first host according to the starting address and the memory size.

7. An automatic analysis device for AI accelerator card calculation errors, characterized in that: The AI ​​accelerator card calculation error automatic analysis device comprises: an acquisition module, configured to acquire a first data stream and a second data stream, wherein the first data stream is a data stream between a first host and a first device in a first AI accelerator card when the two devices perform a computing task, and the second data stream is a data stream between a second host and a second device in a second AI accelerator card when the two devices perform the computing task; A comparison module, used for comparing the data belonging to the same subtask in the first data stream and the second data stream to obtain an error; The acquisition module is used to: detect the call of a transmission function, where the transmission function is used to transmit data between the first host and the first device; when detecting that the transmission function is called, intercept the parameters transmitted to the transmission function; and capture the data transmitted between the first host and the first device according to the parameters to obtain the first data stream; Among them, capturing the data transmitted between the first host and the first device according to the parameters to obtain the first data stream includes: obtaining the data transmission stage between the first host and the first device; if the data transmission stage is a stage in which the first host transmits data to the first device, determining the starting address and the memory size according to the parameters, and obtaining memory data in the memory of the first host according to the starting address and the memory size.

8. An electronic device, characterized in that: include: A memory and a processor, wherein the processor is connected to the memory; The memory is used to store programs; The processor is used to call the program stored in the memory to execute the method according to any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is executed.

Images