Accelerator card-based reasoning task detection system and method

Abstract

The invention provides a reasoning task detection system and method based on an accelerator card, and the system comprises an interaction unit which is used for receiving a task code of a reasoning task of a user system, and the task code is obtained based on the calling of a target operator; the compiling unit is used for linking the task code with the operator library and compiling the task code to obtain a to-be-run task file; the operator library is stored in the acceleration card; and the detection unit is used for executing the to-be-run task file based on the accelerator card. According to the system provided by the invention, a task code of a reasoning task of a user system is received; the task code is linked with the operator library stored in the acceleration card, the to-be-run task file is obtained through compiling, and the to-be-run task file is executed based on the acceleration card, so that simple, convenient and easy-to-master task code compiling is realized, a user system does not need to compile the task code, and the compiled to-be-run task file can directly run on the acceleration card; the efficiency and usability of reasoning task detection are greatly improved, and plug-and-play reasoning task detection is achieved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an accelerator card-based reasoning task detection system and method. Background Art

[0002] The rapid development of artificial intelligence (AI), the surge in computing power, and the pursuit of efficient computing have led to the rise of accelerators such as Tensor Processing Units (TPUs), GPUs, and ASICs. TPUs efficiently support deep learning inference with their tensor processing units, GPUs empower complex model training with massively parallel computing units, and ASICs achieve extreme energy efficiency in specific scenarios through customized architectures. These three technologies form a complementary "training-inference-dedicated" ecosystem.

[0003] However, existing acceleration solutions generally have problems such as high development thresholds, fragmented programming models, and complex cross-platform adaptation. Developers need to have an in-depth understanding of hardware architecture details, making it difficult to quickly transform algorithm innovations into actual productivity. Summary of the Invention

[0004] The present invention provides an accelerator-based reasoning task detection system and method, which are used to solve the common defects of the prior art acceleration detection methods for reasoning tasks, such as high development threshold, fragmented programming model, and complex cross-platform adaptation.

[0005] The present invention provides an inference task detection system for an accelerator card, comprising: an interaction unit, a compilation unit, and a detection unit, wherein a communication connection is established among the interaction unit, the compilation unit, and the detection unit; The interaction unit is configured to receive a task code of an inference task of a user system, where the task code is obtained by calling a target operator corresponding to the inference task; The compilation unit is used to link the task code with the operator library, compile the task code, and obtain a task file to be run; the operator library includes multiple target operators, and the operator library is stored in the accelerator card; The detection unit is configured to execute the task file to be run based on the accelerator card.

[0006] According to an accelerator card-based reasoning task detection system provided by the present invention, the target operator is declared in the form of a header file.

[0007] According to an accelerator card-based reasoning task detection system provided by the present invention, the compilation unit is specifically configured to: Linking the task code with the operator library, compiling the task code, and generating a compilation result code; Returning the compilation result code to the user system, and receiving an update task code and an update target operator from the user system if the compilation result code indicates a compilation failure; the update target operator is used to update the operator library; The updated task code and the updated operator library are relinked, and the updated task code is compiled to obtain the task file to be run.

[0008] According to an accelerator card-based reasoning task detection system provided by the present invention, the detection unit is specifically configured to: receiving inference data from the user system; Based on the accelerator card, applying the inference data, executing the task file to be run, and returning the actual inference result of the task file to be run to the user system; The inference data includes input inference data and expected inference results.

[0009] According to the present invention, an accelerator card-based reasoning task detection system further includes a resource management unit, which is specifically configured to: Based on the accelerator card, applying the inference data, executing the task file to be run, and in the process of executing the task file to be run, obtaining at least one of current memory data, current task queue, and current computing resource status of the accelerator card as progress feedback data; The progress feedback data is returned to the user system.

[0010] According to the accelerator card-based reasoning task detection system provided by the present invention, the interaction unit is further specifically configured to: receiving a first instruction from the user system; Based on the first instruction, the to-be-executed task file is sent to the user system.

[0011] According to an accelerator card-based reasoning task detection system provided by the present invention, the interaction unit is implemented based on a PCIE module, and the PCIE module supports hot plugging to access the user system.

[0012] The present invention also provides an accelerator card-based reasoning task detection method, comprising: Receiving a task code of an inference task of a user system, wherein the task code is obtained by calling a target operator corresponding to the inference task; Linking the task code with an operator library, compiling the task code, and obtaining a task file to be run; the operator library includes multiple target operators, and the operator library is stored in an accelerator card; Based on the accelerator card, the task file to be run is executed.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the inference task detection method based on the accelerator card as described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program implements any of the above-described methods for detecting inference tasks based on an accelerator card.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned methods for detecting reasoning tasks based on an accelerator card.

[0016] The present invention provides an accelerator-based reasoning task detection system and method, which receives the task code of the user system's reasoning task through an interactive unit, where the task code is obtained based on calling the target operator corresponding to the reasoning task; links the task code with the operator library stored in the accelerator card through a compilation unit, compiles the task code, and obtains a task file to be run; and executes the task file to be run based on the accelerator card through a detection unit, thereby realizing simple and easy-to-use task code writing, and does not require the user system to compile the task code. The compiled task file to be run can be directly run on the accelerator card, which greatly improves the efficiency and ease of use of reasoning task detection and realizes plug-and-play reasoning task detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 This is one of the structural diagrams of the accelerator card-based reasoning task detection system provided by the present invention; Figure 2 This is the second structural diagram of the accelerator card-based reasoning task detection system provided by the present invention; Figure 3 It is a schematic diagram of the workflow of the scheduling system provided by the present invention; Figure 4 This is one of the flow charts of the accelerator card-based reasoning task detection method provided by the present invention; Figure 5 This is the second flow chart of the accelerator card-based reasoning task detection method provided by the present invention; Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0019] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0020] In view of the above problems, the present invention provides an accelerator card-based reasoning task detection system to provide a concise and easy-to-use reasoning task detection device. Figure 1 This is one of the structural diagrams of the inference task detection system based on the accelerator card provided by the present invention, such as Figure 1 As shown, the system includes: an interaction unit 110, a compilation unit 120 and a detection unit 130, wherein communication connections are established among the interaction unit 110, the compilation unit 120 and the detection unit 130; The interaction unit 110 is configured to receive a task code of an inference task of a user system, where the task code is obtained by calling a target operator corresponding to the inference task; Here, the user system refers to the host, such as a server CPU, which interacts with the accelerator card through a standardized interface. The inference task here refers to a model that uses a trained model, such as a deep learning model, to predict or analyze input inference data. For example, an inference task can be a model used for object detection. Therefore, the inference task detection process can be considered as using the trained model corresponding to the inference task to predict or analyze the input inference data, and the prediction or analysis results are used to test the performance of the model for the inference task.

[0021] In this context, the task code for an inference task refers to the code that constitutes the model corresponding to the inference task. The target operator refers to the predefined AI computational unit corresponding to the inference task, such as convolution or activation functions. It should be noted that the target operator can be encapsulated as a function interface, allowing the task code corresponding to the inference task to be written by calling the target operator.

[0022] Specifically, first, it is possible to interact with the user system through PCIe, and the interaction unit receives the task code of the reasoning task sent by the user system. The task code here can be written by the user calling the target operator corresponding to the reasoning task. It should be noted that the target operator can be expressed in the form of a header file, declaring the interface of the function specifically implemented by the target operator. As a result, the user only needs to describe the calculation process corresponding to the reasoning task by calling the target operator, just like the CPU programming development process, without installing the supporting development kit or understanding the underlying hardware details, to achieve concise and easy-to-use task code writing, which not only lowers the user's usage threshold, but also improves the detection efficiency of the reasoning task.

[0023] The compiling unit 120 is used to link the task code with the operator library, compile the task code, and obtain a task file to be executed; the operator library includes multiple target operators, and the operator library is stored in the accelerator card; Here, an operator library refers to a resource library containing multiple target operators. This library can include target operators corresponding to multiple inference tasks, enabling detection of multiple inference tasks. The operator library can be stored in the accelerator card's storage unit and can communicate with the user system via the PCIe interface, receiving new target operators issued by the user system.

[0024] In addition, the task file to be run here refers to the file of the inference task that can be run directly, which is used to detect whether the inference task can achieve the expected prediction effect.

[0025] Specifically, the task code can be sent to the compilation unit, which schedules the compilation unit, links the task code with the operator library, and compiles the task code to obtain the task file to be executed. It should be noted that by using the pre-stored operator library, after obtaining the task code, there is no need to compile the task code on the user system side, which saves the user system's computing resources. By utilizing the high-computing accelerator card to compile the task code, the compilation efficiency is greatly improved, thereby improving the detection efficiency of the inference task.

[0026] The detection unit 130 is configured to execute the task file to be run based on the accelerator card.

[0027] Specifically, the accelerator card can execute the task file to be run, obtain the corresponding output result, and then determine whether the output result is consistent with the expected result to achieve the detection of the reasoning task. It should be noted that by utilizing the high performance of the accelerator card to execute the task file to be run, the efficiency of reasoning task detection is greatly improved.

[0028] It should be noted that it is possible to connect to the user system through a standardized interface, such as the PCIe interface, to achieve plug-and-play reasoning task detection, greatly improving the simplicity of reasoning task detection.

[0029] The system provided by the embodiment of the present invention receives the task code of the inference task of the user system through the interaction unit, and the task code is obtained based on calling the target operator corresponding to the inference task; the task code is linked to the operator library stored in the accelerator card through the compilation unit, and the task code is compiled to obtain the task file to be run. The task file to be run is executed based on the accelerator card through the detection unit, thereby realizing simple and easy-to-use task code writing, and there is no need for the user system to compile the task code. The compiled task file to be run can be directly run on the accelerator card, which greatly improves the efficiency and ease of use of inference task detection and realizes plug-and-play inference task detection.

[0030] Based on any of the above embodiments, the target operator is declared in the form of a header file.

[0031] Specifically, the target operator can be declared in the form of a header file, which declares the interface, macros, constants, and data types of the function implemented by the target operator. This allows the functions related to the inference task to be declared in a single header file. This allows users to describe the computational process of the model corresponding to the inference task by simply calling the target operator, just as they would during CPU programming development. This allows for easy and user-friendly task code development, simplifying the writing process.

[0032] In addition, representing the target operator in the form of a header file can achieve modular design. Different modules can design corresponding header files to achieve the separation and organization of task codes, which makes it easier to maintain and manage the task codes in the future.

[0033] Based on any of the above embodiments, the compiling unit 120 is specifically configured to: Linking the task code with the operator library, compiling the task code, and generating a compilation result code; Returning the compilation result code to the user system, and receiving an update task code and an update target operator from the user system if the compilation result code indicates a compilation failure; the update target operator is used to update the operator library; The updated task code and the updated operator library are relinked, and the updated task code is compiled to obtain the task file to be run.

[0034] Specifically, after obtaining the task code, you can link the task code with the operator library, compile the task code using the corresponding target operator in the operator library, and generate a compilation result code. The compilation result code here is used to reflect whether the task code compilation is successful, and the compilation result code is returned to the user system.

[0035] Furthermore, when the compilation result code indicates that the compilation failed, it means that there may be a writing error in the task code, or there may be a missing or wrong target operator in the operator library. Therefore, the updated target operator can be updated to the operator library by receiving the updated task code and updated target operator issued by the user system. It should be noted that the updated target operator can be added to, or replaced by the updated operator library with the corresponding original target operator in the operator library, and the selection can be made according to the actual situation. It can be understood that when the compilation result code indicates that the compilation is successful, a successfully compiled task file to be run can be obtained.

[0036] Next, the updated task code and the updated operator library are relinked, the updated task code is compiled, and the task file to be executed is obtained. The compilation result code is returned to the user system. It should be noted that if the compilation result code here still indicates failure, the updated task code and updated target operator from the user system are continuously received and the task code is recompiled until the compilation succeeds and the task file to be executed is obtained.

[0037] The system provided by the embodiment of the present invention completes the compilation process of the task code through the reasoning task detection system. In the event of a failure in the compilation process, the updated task code and updated target operator of the user system are received through the established communication link between the user system and the reasoning task detection system, thereby completing the successful compilation of the task code and obtaining the task file to be run, thereby realizing a concise compilation process and improving the overall simplicity of reasoning task detection.

[0038] Based on any of the above embodiments, the detection unit 130 is specifically configured to: receiving inference data from the user system; Based on the accelerator card, applying the inference data, executing the task file to be run, and returning the actual inference result of the task file to be run to the user system; The inference data includes input inference data and expected inference results.

[0039] Here, inference data refers to the input inference data and expected inference results used for inference task detection.

[0040] Specifically, the system can receive inference data transmitted by the user system, which includes input inference data and expected inference results. For example, when the inference task is an object detection task, the input inference data can be an image of a puppy, and the expected inference result can be "puppy".

[0041] It should be noted that when testing reasoning tasks, to verify the performance of the reasoning task model, a large amount of pre-built reasoning data of different types can be used to complete the reasoning task testing. The reasoning data here can come from sample data during the training process of the reasoning task model. The sample input can be used as the input reasoning data, and the sample label can be used as the expected reasoning result corresponding to the input reasoning data.

[0042] Furthermore, the input inference data in the inference data can be used as input data for the task file to be executed. The task file to be executed is executed by the accelerator card to obtain the actual inference result, which can be returned to the user system. It is understood that if the actual inference result is consistent with the expected inference result corresponding to the input inference data, it indicates that the model of the inference task is highly correct.

[0043] It should be noted that the inference task detection system, which includes a standardized interface, facilitates fast and convenient data transmission between the user system and the inference task detection system, thereby improving the convenience and efficiency of inference task detection. Furthermore, by using a high-performance accelerator card, performance testing of inference tasks using inference data can be achieved. This eliminates the need for cross-platform collaboration between TPUs, GPUs, and ASICs as in the prior art. Instead, the inference task detection system provided by the embodiments of the present invention enables "one-stop" inference task detection.

[0044] Based on any of the above embodiments, the reasoning task detection system further includes a resource management unit, which is specifically configured to: Based on the accelerator card, applying the inference data, executing the task file to be run, and in the process of executing the task file to be run, obtaining at least one of current memory data, current task queue, and current computing resource status of the accelerator card as progress feedback data; The progress feedback data is returned to the user system.

[0045] Specifically, the input inference data in the inference data can be used as the input data for the task file to be executed. The computing unit in the accelerator card, such as the AI Engine unit, can calculate the input data according to the inference task model. During the execution of the task file to be executed, that is, during the model calculation process, progress feedback data can be obtained and returned to the user system.

[0046] It should be noted that the progress feedback data here can be at least one of the current memory data, current task queue, and current computing resources of the accelerator card. Among them, the current memory data and current computing resources can be used to reflect the resource occupancy of the subtask corresponding to the file to be run that is currently being executed by the accelerator card. It can be understood that if the occupancy rate of the current memory data and the current computing resources is high, it means that the resource occupancy of the subtask being executed is large, which can reflect that the computing energy consumption of the model corresponding to the reasoning task is high; if the occupancy rate of the current memory data and the current computing resources is low, it means that the resource occupancy of the subtask being executed is large, which can reflect that the computing energy consumption of the model corresponding to the reasoning task is high.

[0047] In addition, the current task queue can reflect the execution time of each subtask corresponding to the task file to be run, thereby reflecting the computational efficiency of the task. When testing the inference task, the model's performance indicators can include at least one of accuracy, computational efficiency, and computational energy consumption. Therefore, by returning progress feedback data to the user system, the user system can conduct a more comprehensive test of the performance of the model corresponding to the inference task based on the progress feedback data, that is, to test the computational efficiency and computational energy consumption performance, thereby facilitating the optimization of the model corresponding to the inference task.

[0048] Based on any of the above embodiments, the interaction unit is further specifically configured to: receiving a first instruction from the user system; Based on the first instruction, the to-be-executed task file is sent to the user system.

[0049] Specifically, a first instruction from a user system may be received. The first instruction may be an instruction for instructing the inference task detection system to send a pending task file to the user system, such as a request to obtain a pending task file. Then, according to the first instruction, the pending task file is sent to the user system for storage by the user for subsequent use.

[0050] Based on any of the above embodiments, the interaction unit is implemented based on a PCIE module, and the PCIE module supports hot plugging to access the user system.

[0051] Here, the interaction unit can be implemented as a PCIE module, which is used to enable data transmission between the user system and the inference task detection system. It should be noted that the PCIE interface is hot-swappable and can be directly connected to the user system, thereby enabling plug-and-play inference task detection. In practical applications, the PCIE interface can be connected to any user system to establish a communication link with the user system, enabling simple and easy-to-use inference task detection.

[0052] Figure 2This is the second structural diagram of the inference task detection system based on the accelerator card provided by the present invention, such as Figure 2 As shown in the figure, the system includes: PCIE interface, Boot ROM module, Lib ROM module, RISCV core module, AI Engine module, DDR module, and Ethernet module. Among them, the PCIE interface supports hot-swappable function to connect to the user system and allows users to query the execution status of tasks and system resource usage. The Boot ROM module is used to store the scheduling system on the accelerator card, monitor the device memory, task queue, computing unit status and other information in real time, and provide necessary feedback. The Lib ROM module is used to store the programmable interface library, that is, to store the operator library. The RISCV core module is used to run the scheduling system on the accelerator card, integrating multiple scheduling strategies including but not limited to priority strategy and polling scheduling according to computing resources and real-time requirements. The AIEngine module is used to execute the target operator. The DDR module refers to the large-capacity storage interface. The Ethernet module refers to the high-speed network interface, which is used for efficient data transmission between different modules. It also provides programmable cascade interface services, provides basic clustering functions, and realizes cluster deployment.

[0053] Specifically, it interacts with the user system via the PCIE interface. Unlike previous accelerator cards, it eliminates the need for a development kit. Customized target operators are provided as header files. Users simply describe the computational process of the model corresponding to the inference task by calling the customized target operator, just as they would during CPU programming. After obtaining the task code, the user directly sends the programmed model to the accelerator card, eliminating the need for host-side compilation. The accelerator card compiles and checks the model's task code through a scheduling system. The compiled task file can then be run directly on the accelerator card. Users can also retrieve the task file through the PCIE interface and save it for later use.

[0054] In actual applications, after the device corresponding to the inference task detection system is powered on, the RISCV core loads and runs the scheduling system from the Boot ROM. The PCIE module completes the communication link between the device and the host. User data is sent to the device-side DDR through PCIe. After the user task is processed by the scheduling system, the interface library stored in the Lib ROM sends the computing instructions to AIEngine.

[0055] in, Figure 3 This is a schematic diagram of the workflow of the scheduling system provided by the present invention. Figure 3As shown in Figure 1, the process involves the following: First, after the scheduling system starts, the communication and interconnection module (Ethernet module) completes device initialization. Next, the resource management module (Boot ROM module) provides resource information to the task management module (AI Engine module) and the scheduling module (RISCV core module). The task management module waits for the user interface module (PCIE interface) to add tasks to the task queue; the scheduling module selects executable tasks based on the scheduling policy.

[0056] Figure 4 This is one of the flow charts of the inference task detection method based on the accelerator card provided by the present invention, such as Figure 4 As shown, the method includes: Step 410: Receive a task code of an inference task of a user system, where the task code is obtained by calling a target operator corresponding to the inference task; Step 420: linking the task code with an operator library, compiling the task code, and obtaining a task file to be executed; the operator library includes multiple target operators, and the operator library is stored in the accelerator card; Step 430: Execute the task file to be run based on the accelerator card.

[0057] Figure 5 This is the second flow chart of the inference task detection method based on the accelerator card provided by the present invention, such as Figure 5 As shown, the method includes: first, describing the trained model using a provided customized operator. Here, the customized operator refers to the target operator corresponding to the inference task, and the trained model refers to the model corresponding to the inference task. Then, the operator code is sent to the accelerator card through the device interface. Here, the operator code refers to the task code.

[0058] Furthermore, the device links the operator code with a customized operator library, generates an executable model file, and provides feedback. The customized operator library is the operator library described in any of the above embodiments, and the executable model file is the task file to be run. The feedback here indicates whether the task code was compiled successfully.

[0059] Next, the model data is distributed and the calculation process is initiated. Specifically, the system receives inference data from the user system, executes the task file to be run, and implements the inference task detection. Finally, the device completes the calculation and returns the results.

[0060] The method provided by the embodiment of the present invention receives the task code of the reasoning task of the user system through the interaction unit, and the task code is obtained based on calling the target operator corresponding to the reasoning task; the task code is linked to the operator library stored in the accelerator card through the compilation unit, and the task code is compiled to obtain the task file to be run. The task file to be run is executed based on the accelerator card through the detection unit, thereby realizing simple and easy-to-use task code writing, and there is no need for the user system to compile the task code. The compiled task file to be run can be directly run on the accelerator card, which greatly improves the efficiency and ease of use of reasoning task detection and realizes plug-and-play reasoning task detection.

[0061] Based on any of the above embodiments, the target operator is declared in the form of a header file.

[0062] Based on any of the above embodiments, linking the task code with the operator library, compiling the task code, and obtaining a task file to be executed includes: Linking the task code with the operator library, compiling the task code, and generating a compilation result code; Returning the compilation result code to the user system, and receiving an update task code and an update target operator from the user system if the compilation result code indicates a compilation failure; the update target operator is used to update the operator library; The updated task code and the updated operator library are relinked, and the updated task code is compiled to obtain the task file to be run.

[0063] Based on any of the foregoing embodiments, executing the to-be-run task file based on the accelerator card includes: receiving inference data from the user system; Based on the accelerator card, applying the inference data, executing the task file to be run, and returning the actual inference result of the task file to be run to the user system; The inference data includes input inference data and expected inference results.

[0064] Based on any of the above embodiments, the method further includes: Based on the accelerator card, applying the inference data, executing the task file to be run, and in the process of executing the task file to be run, obtaining at least one of current memory data, current task queue, and current computing resource status of the accelerator card as progress feedback data; The progress feedback data is returned to the user system.

[0065] Based on any of the above embodiments, the method further includes: receiving a first instruction from the user system; Based on the first instruction, the to-be-executed task file is sent to the user system.

[0066] Based on any of the above embodiments, the interaction unit is implemented based on a PCIE module, and the PCIE module supports hot plugging to access the user system.

[0067] Figure 6 An example of a physical structure diagram of an electronic device is shown below. Figure 6 As shown, the electronic device may include: a processor 610, a communications interface 620, a memory 630, and a communications bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other via the communications bus 640. The processor 610 may call logic instructions in the memory 630 to execute an accelerator card-based reasoning task detection method, which includes: receiving a task code for a reasoning task of a user system, the task code being obtained by calling a target operator corresponding to the reasoning task; linking the task code with an operator library, compiling the task code, and obtaining a task file to be executed; the operator library includes multiple target operators, and the operator library is stored in the accelerator card; and executing the task file to be executed based on the accelerator card.

[0068] Furthermore, the logic instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0069] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the reasoning task detection method based on the accelerator card provided by the above methods. The method includes: receiving the task code of the reasoning task of the user system, and the task code is obtained based on calling the target operator corresponding to the reasoning task; linking the task code with the operator library, compiling the task code, and obtaining a task file to be run; the operator library includes multiple target operators, and the operator library is stored in the accelerator card; based on the accelerator card, executing the task file to be run.

[0070] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the reasoning task detection method based on the accelerator card provided by the above methods. The method includes: receiving the task code of the reasoning task of the user system, and the task code is obtained based on calling the target operator corresponding to the reasoning task; linking the task code with the operator library, compiling the task code, and obtaining a task file to be run; the operator library includes multiple target operators, and the operator library is stored in the accelerator card; based on the accelerator card, executing the task file to be run.

[0071] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0072] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. An accelerator card-based reasoning task detection system, characterized in that: include: An interaction unit, a compilation unit, and a detection unit, wherein a communication connection is established among the interaction unit, the compilation unit, and the detection unit; The interaction unit is configured to receive a task code of an inference task of a user system, where the task code is obtained by calling a target operator corresponding to the inference task; The compilation unit is used to link the task code with the operator library, compile the task code, and obtain a task file to be run; the operator library includes multiple target operators, and the operator library is stored in the accelerator card; The detection unit is configured to execute the task file to be run based on the accelerator card.

2. The reasoning task detection system based on the accelerator card according to claim 1, characterized in that: The target operator is declared in the form of a header file.

3. The reasoning task detection system based on the accelerator card according to claim 1, characterized in that: The compilation unit is specifically used for: Linking the task code with the operator library, compiling the task code, and generating a compilation result code; Returning the compilation result code to the user system, and receiving an update task code and an update target operator from the user system if the compilation result code indicates a compilation failure; the update target operator is used to update the operator library; The updated task code and the updated operator library are relinked, and the updated task code is compiled to obtain the task file to be run.

4. The reasoning task detection system based on an accelerator card according to any one of claims 1 to 3, characterized in that: The detection unit is specifically used for: receiving inference data from the user system; Based on the accelerator card, applying the inference data, executing the task file to be run, and returning the actual inference result of the task file to be run to the user system; The inference data includes input inference data and expected inference results.

5. The reasoning task detection system based on the accelerator card according to claim 4, characterized in that: The reasoning task detection system further includes a resource management unit, which is specifically configured to: Based on the accelerator card, applying the inference data, executing the task file to be run, and in the process of executing the task file to be run, obtaining at least one of current memory data, current task queue, and current computing resource status of the accelerator card as progress feedback data; The progress feedback data is returned to the user system.

6. The reasoning task detection system based on an accelerator card according to any one of claims 1 to 3, characterized in that: The interaction unit is further specifically configured to: receiving a first instruction from the user system; Based on the first instruction, the to-be-executed task file is sent to the user system.

7. The reasoning task detection system based on an accelerator card according to any one of claims 1 to 3, characterized in that: The interaction unit is implemented based on a PCIE module, and the PCIE module supports hot plugging to access the user system.

8. A method for detecting reasoning tasks based on an accelerator card, characterized in that: include: Receiving a task code of an inference task of a user system, wherein the task code is obtained by calling a target operator corresponding to the inference task; Linking the task code with an operator library, compiling the task code, and obtaining a task file to be run; the operator library includes multiple target operators, and the operator library is stored in an accelerator card; Based on the accelerator card, the task file to be run is executed.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the inference task detection method based on the accelerator card as claimed in claim 8 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for detecting reasoning tasks based on an accelerator card as claimed in claim 8 is implemented.

Images