Test task execution method and task compilation method of artificial intelligence model, chip, device and computer equipment

Abstract

This application belongs to the field of computer technology and provides a method for executing test tasks of artificial intelligence models, a task compilation method, a chip, a device, and a computer device. This application pre-compiles the task to be processed of the artificial intelligence model, so that it has multiple sub-tasks that achieve performance balance. Each sub-test task in the self-test task set is assigned to the corresponding sub-task of the task to be processed. During the execution of each sub-task by the NPU of the computer device running the target application related to artificial intelligence technology, the sub-test tasks assigned to that sub-task can be executed synchronously. This realizes the efficient utilization of NPU resources and the parallel execution of self-test function safety checks. Unlike conventional technologies, it is not necessary to execute all the sub-test tasks in the self-test task set after completing a complete inference task, thereby improving the effective computing power and resource utilization of the NPU.

Description

Technical Field

[0001] This application belongs to the field of computer technology, and in particular relates to a method for executing test tasks of an artificial intelligence model, a task compilation method, a chip, a device, and a computer equipment. Background Technology

[0002] During the reasoning process of computer devices performing artificial intelligence (AI) related tasks, the NPU (Neural Processing Unit) is responsible for accelerating the reasoning of various AI models to complete the tasks required by the application. To ensure system security, self-testing techniques, especially software test libraries (STL), are often used as a mainstream low-cost software solution to periodically detect random hardware faults (RHWF) during system operation.

[0003] In conventional technical solutions, model inference tasks and test tasks in the self-test task set are executed sequentially. That is, the NPU is only allowed to execute an STL test task after completing an AI model inference, and only after that test task is completed can it continue to execute other AI model inference tasks. This approach limits the utilization of NPU hardware resources. Summary of the Invention

[0004] This application provides a method for executing test tasks of an artificial intelligence model, a task compilation method, a chip, a device, and a computer equipment. It can solve the technical problem that current conventional technologies limit the computing power and resource utilization of NPUs when running application software based on artificial intelligence technology. This is because a complete inference task must be executed before all sub-test tasks in the self-test task set can be executed.

[0005] First, to achieve the above objectives, this application provides a test task execution method for an artificial intelligence model. The test task execution method is applied to a chip, which includes an NPU. The chip is deployed on a computer device storing a target application and a set of self-test tasks. When the computer device runs the target application, the NPU is configured to invoke an artificial intelligence model to perform inference on a task to be inferred from the target application. The set of self-test tasks is used to perform system testing on the task to be inferred, and the set of self-test tasks includes several sub-test tasks.

[0006] The task to be inferred is divided into N subtasks according to a preset compilation method, and each sub-test task is assigned to a subtask according to the preset compilation method. The subtasks are arranged in subtask order, each subtask contains a network layer, and the network layers in the subtasks are arranged in network layer order. Each network layer belongs to one of the artificial intelligence models. The task execution method includes:

[0007] The NPU invokes the artificial intelligence model and sequentially traverses the N subtasks of the task to be reasoned according to the order of the subtasks, and calls m computing resources to process the network layer in each subtask in order to perform task reasoning on the task to be reasoned, where N and m are both positive integers.

[0008] During the process of the NPU calling the m computing resources to process the current network layer in the current subtask, if it is determined that there are idle computing resources among the m computing resources that are no longer processing the current network layer, the NPU will allocate the current sub-test task that has not been executed in the current subtask to the idle computing resources to perform system testing on the current subtask.

[0009] For example, the various subtasks among the N subtasks satisfy a performance balance condition, which at least indicates that the various subtasks occupy the same or similar amounts of the various computing resources among the m computing resources, and that the computational amount of each subtask is the same or similar.

[0010] Secondly, this application also provides a task compilation method for an artificial intelligence model, the compilation method comprising:

[0011] The inference tasks related to the artificial intelligence model are compiled and divided according to the number of network layers to obtain a first undetermined division result. The first undetermined division result includes L sub-tasks, and each sub-task contains several consecutive network layers; where L is a positive integer.

[0012] According to the first preset processing method, several sub-test tasks in the self-test task set are assigned to L sub-tasks in the first undetermined partitioning result to obtain the second undetermined partitioning result;

[0013] The process iterates over each subtask of the second undetermined partitioning result to adjust the number of network layers and the number of subtest tasks in each subtask until the performance balance condition is met between each subtask, and stops the iteration to obtain a target partitioning result that meets the performance balance condition. The target partitioning result includes N subtasks. Each subtask in the target partitioning result is used as the basic scheduling unit of the NPU to complete the inference of the task to be inferred.

[0014] For example, according to a first preset processing method, several sub-test tasks in the self-test task set are assigned to L sub-tasks in the first pending partitioning result to obtain a second pending partitioning result, including:

[0015] The first undetermined partitioning result is simulated to obtain the simulation result;

[0016] The scheduling performance indicators of each subtask in the simulation results are analyzed to obtain the analysis results;

[0017] Based on the analysis results, several sub-test tasks in the self-test task set are assigned to L sub-tasks in the first pending partitioning result to obtain the second pending partitioning result.

[0018] For example, the scheduling performance metrics include at least the runtime of each subtask and the computational resource usage of each subtask.

[0019] To achieve the above objectives, this application provides a chip comprising a CPU and an NPU, deployed on a computer device storing a target application and a set of self-test tasks. The NPU is configured to, when the CPU is running the target application, invoke an artificial intelligence model to perform inference on a task to be inferred in the target application. The set of self-test tasks is used to perform system testing on the task to be inferred, and the set of self-test tasks includes several sub-test tasks. The task to be inferred is divided into N sub-tasks according to a preset compilation method, and each sub-test task is assigned to a sub-task according to the preset compilation method. The sub-tasks are arranged in sub-task order, each sub-task contains a network layer, the network layers of the sub-tasks are arranged in network layer order, and each network layer belongs to one of the artificial intelligence models.

[0020] The NPU is used to call the artificial intelligence model and sequentially traverse the N subtasks of the task to be reasoned according to the order of the subtasks, and call m computing resources to process the network layer in each subtask in order to perform task reasoning on the task to be reasoned.

[0021] In the process of calling the m computing resources to process the current network layer in the current subtask, the NPU is also configured to, if it is determined that there are idle computing resources among the m computing resources that are no longer processing the current network layer, allocate the current sub-test task that has not been executed in the current subtask to the idle computing resources, so as to perform system testing on the current subtask.

[0022] For example, the NPU includes a command distribution unit and a computing unit;

[0023] The command distribution unit is configured to write the configuration information of each subtask of the task to be reasoned into the computing unit;

[0024] The computing unit is used to perform computational processing on the current subtask based on the configuration information and by calling computing resources to obtain the subtask processing result.

[0025] The command distribution unit is configured to allocate the currently unexecuted sub-test task in the current sub-task to the idle computing resources if it is determined that there are idle computing resources in the computing unit that are no longer processing the current sub-task.

[0026] The computing unit is also used to call upon the idle computing resources to run the current sub-test task in order to perform system testing on the current sub-task.

[0027] For example, the computing unit includes at least a first computing resource and a second computing resource;

[0028] The command distribution unit is further configured to, when it is determined that the first computing resource is processing the current sub-task and the second computing resource is no longer processing the current sub-task, allocate the unexecuted current sub-test task in the current sub-task to the second computing resource;

[0029] The computing unit is also used to call the second computing resource to run the current sub-test task in order to perform system testing on the current sub-task.

[0030] To achieve the above objectives, this application also provides a test task execution device based on an artificial intelligence model, the device comprising:

[0031] The subtask execution module is used to call the artificial intelligence model and traverse the N subtasks of the task to be reasoned in the order of the subtasks, and call m computing resources to process the network layer in each subtask to perform task reasoning on the task to be reasoned, where N and m are both positive integers.

[0032] The sub-test task execution module is used to, during the process of calling the m computing resources to process the current network layer in the current sub-task, if it is determined that there are idle computing resources among the m computing resources that are no longer processing the current network layer, allocate the current sub-test task that has not been executed in the current sub-task to the idle computing resources, so as to perform system testing on the current sub-task.

[0033] To achieve the above objectives, this application also provides a computer device, which includes the chip, memory, and test task execution program of an artificial intelligence model stored in the memory and executable on the chip as described above. When the test task execution program of the artificial intelligence model is executed by the chip, it implements the steps of the method described above.

[0034] To achieve the above objectives, this application also provides a computer program product, including a computer program, which, when run, causes the method described above to be performed.

[0035] To achieve the above objectives, this application also provides a vehicle equipped with the computer device described above, the computer device including the chip described above, wherein when the computer device runs the target application, the chip performs the steps of the method described above.

[0036] The beneficial effects of this application are as follows: it can pre-compile the task to be processed of the artificial intelligence model, so that it has multiple sub-tasks that achieve performance balance, and each sub-test task in the self-test task set is assigned to the corresponding sub-task of the task to be processed. During the execution of each sub-task by the NPU of the computer device of the target application related to artificial intelligence technology, the sub-test tasks assigned to that sub-task can be executed synchronously, realizing the efficient utilization of NPU resources and the parallel execution of self-test function safety checks. Unlike conventional technologies, it is not necessary to execute all the sub-test tasks in the self-test task set after completing a complete inference task, thereby improving the effective computing power and resource utilization of the NPU. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a structural block diagram of a computer device provided in an embodiment of this application;

[0039] Figure 2 This is a schematic diagram of an embodiment of the compilation stage of the artificial intelligence model task provided in this application;

[0040] Figure 3 This is a schematic flowchart of an embodiment of a test task execution method for an artificial intelligence model provided in this application;

[0041] Figure 4 This is a schematic diagram of the structure of the NPU chip provided in the embodiments of this application;

[0042] Figure 5 This is a schematic flowchart of another embodiment of the test task execution method for an artificial intelligence model provided in this application;

[0043] Figure 6 This application provides a schematic flowchart of an embodiment of a task compilation method for an artificial intelligence model.

[0044] Figure 7 This is a schematic flowchart of another embodiment of the task compilation method for an artificial intelligence model provided in this application;

[0045] Figure 8 This is a block diagram of the compilation stage architecture provided in one embodiment of this application;

[0046] Figure 9 This is a schematic diagram of the structure of the task execution device provided in the embodiments of this application. Detailed Implementation

[0047] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0048] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0049] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0050] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0051] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0052] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0053] Understandably, the inventors of this application have found that in order to ensure system security, self-testing technology, especially the software testing library STL, is often used as a mainstream low-cost software solution to periodically detect random hardware failures (RHWF) during the operation of computer devices running application software related to artificial intelligence technology.

[0054] For example, in the field of autonomous driving, the NPU (Neural Processing Unit) of an AI-powered autonomous driving terminal is responsible for accelerating the inference of various neural network models to complete various intelligent driving tasks such as road condition perception and navigation assistance. Each intelligent driving task requires an automatic test after completion. In conventional implementations, model inference tasks and STL tasks are executed sequentially; that is, the NPU is only allowed to execute an STL task after completing a model inference, and only after the STL task is completed can the model inference task continue. This approach limits the utilization of NPU hardware resources: STL, as a periodically running safety-assurance task, significantly occupies additional NPU runtime.

[0055] To address the technical problem that current conventional technologies limit the computing power and resource utilization of NPUs when running AI-based application software, as a complete inference task must be executed before all sub-test tasks in the self-test task set can be executed, this application provides a method, chip, device, and computer equipment for executing test tasks of an AI model. The technical solution of this application will be illustrated below through specific embodiments.

[0056] Firstly, in a first aspect, embodiments of this application provide a method for executing a test task of an artificial intelligence model, and a chip for executing the method; the chip in this embodiment can be a system-on-chip (SoC), including at least a CPU (Central Processing Unit) and an NPU (Neural Processing Unit); such as Figure 1 As shown, the chip (20) in this embodiment is deployed on a computer device storing a target application related to artificial intelligence technology. The computer device is also provided with a memory 21. In some embodiments, the memory 21 may be an internal storage unit of the computer device, such as a hard disk or memory. In other embodiments, the memory 21 may also be an external storage device of the computer device, such as a dynamic random access memory (DRAM), a solid state drive (SSD), or a hard disk drive (HDD) provided on the computer device. The memory 21 is used to store the operating system, application programs, bootloader, data, and other programs. The memory 21 of the computer device stores a target application 22 and a set of self-test tasks 23. The CPU (201) is configured to run the target application 22 and place the inference tasks in the target application 22 into a task queue.

[0057] When the computer device is running the target application 22, the NPU (202) is configured to invoke an artificial intelligence model to infer the tasks to be inferred in the task queue; wherein, in this embodiment of the application, the task to be inferred of the running target application 22 can be divided into multiple sub-tasks Partitions according to the technical solution of the embodiment of the preset compilation method (the second aspect below), and each sub-test task of the self-test task set 23 is divided into each sub-task Partition according to the preset compilation method; the sub-tasks are arranged in sub-task order, each sub-task contains a network layer, the network layers of the sub-tasks are arranged in network layer order, and each network layer belongs to one of the artificial intelligence models;

[0058] Understandably, if the task queue includes multiple tasks to be reasoned (including Model#1, Model#2, Model#3, etc.), and each task is reasoned by an artificial intelligence model (in this embodiment, a "neural network model" is used as an example), then each task to be reasoned corresponds to a "neural network model". Each task to be reasoned is divided into multiple sub-tasks (Partitions), and each sub-task (Partition) contains several consecutive network layers (Layer), and the sub-tasks (Partitions) are arranged in the order of the sub-tasks (Partition#1, Partition#2, ..., Partition#N); the network layers between the sub-tasks are arranged in the order of the network layers (Layer#1, Layer#2, Layer#3, ..., Layer#N).

[0059] refer to Figure 2Taking a certain inference task Model in the task queue as an example, the Model has three network layers (Layer#1, Layer#2, Layer#3), and there are two sub-test tasks (Test#1, Test#2) in the self-test task set. In this embodiment, the artificial intelligence model Model is divided into two Partitions (Partition#1 and Partition#2) according to the preset compilation method. Partition#1 and Partition#2 are arranged in order. Partition#1 includes a network layer Layer#1 and is inserted into the sub-test task Test#1. Partition#2 includes consecutive network layers Layer#2 and Layer#3 and is inserted into the sub-test task Test#2. In this case, the sub-task Partitions of the Model meet the performance balance condition. The performance balance condition can characterize that the occupancy of different computing resources of the NPU by each sub-task Partition is the same or close, and the computational amount of each sub-task Partition is the same or close.

[0060] Before running the model inference task, the NPU loads the commands corresponding to the subtask partition results generated according to the preset compilation method into the NPU's command dispatcher. The command dispatcher then controls the operation of each submodule of the NPU during the model execution process.

[0061] During the runtime phase, the NPU's command dispatch unit loads each subtask partition sequentially and executes them in order. If the currently executed subtask partition contains an STL subtest task, the NPU, in addition to executing the current network layer within the current subtask for normal inference, will also utilize idle resources to execute the corresponding current subtest task STL.

[0062] Understandably, the target application can be autonomous driving application software, VR (Virtual Reality) racing game application software, or other application software (APP) that requires the use of an artificial intelligence model. This application embodiment uses autonomous driving application software as the target application software running on the computing device as an example for explanation. Correspondingly, the computer device is described as an in-vehicle terminal. During vehicle operation, the in-vehicle terminal is automatically started and the autonomous driving application software is run. During normal vehicle operation, the vehicle will perform a series of routine tasks, including environmental perception and path planning. These tasks to be reasoned will be placed into the task queue (job queue) in order of task priority, and these tasks to be reasoned will be reasoned by the artificial intelligence model in order of task priority.

[0063] Accordingly, refer to Figure 3 The test task execution method of the artificial intelligence model in this application embodiment mainly includes the following steps B:

[0064] Step B: The NPU calls the artificial intelligence model and traverses the N subtasks of the task to be reasoned in order of subtask sequence, and calls m computing resources to process the network layer in each subtask in order to perform task reasoning on the task to be reasoned.

[0065] In the process of the NPU calling m computing resources to process the current network layer in the current subtask, if it is determined that there are idle computing resources among the m computing resources that are no longer processing the current network layer, the NPU will allocate the current sub-test task that has not been executed in the current subtask to the idle computing resources in order to perform system testing on the current subtask.

[0066] The beneficial effects of this application's embodiments are as follows: the task to be processed of the artificial intelligence model is pre-compiled to give it multiple sub-task partitions, and each sub-test task in the self-test task set is assigned to the corresponding sub-task partition of the task to be processed of the artificial intelligence model. During the execution of each sub-task partition by the NPU of the computer device running the target application related to artificial intelligence technology, the sub-test tasks assigned to that sub-task partition can be executed synchronously. This achieves efficient utilization of NPU resources and parallel execution of self-test function safety checks. Unlike conventional technologies, it is not necessary to execute all the sub-test tasks in the self-test task set after completing a complete inference task, thereby improving the effective computing power and resource utilization of the NPU.

[0067] In this embodiment, as Figure 4As shown, the NPU of the chip 20 includes a command dispatch unit 11, a computing unit 12, a memory access module 14, and an on-chip memory 13;

[0068] In a specific implementation, the on-chip memory 13 can be a static random-access memory (SRAM) located inside the NPU chip;

[0069] The memory access module 14 can be a direct memory access module, which is the data transfer module of the NPU and can be used to transfer data in batches between the memory 21 of the computer device and the on-chip registers (SRAM) of the NPU.

[0070] The computing unit 12 may include a first computing resource 121 and a second computing resource 122. For example, the first computing resource 121 may be a matrix multiplication processing core processor (MMP Core) used to accelerate important operators (matrix multiplication) in deep learning; the second computing resource 122 may be a vector-scalar processing core processing unit (VSP Core) for processing vectors and scalars, which is a major computing unit of the NPU used to accelerate various operators (vectors and scalars) in deep learning other than matrix multiplication, such as pooling, residual (ReLU), normalization and other operators.

[0071] Command dispatch unit 11 can be a command dispatcher used to configure the various modules of the NPU; for example, the command dispatch unit writes the configuration information of all currently executed partitions generated during the compilation phase to the first computing resource 121, the second computing resource 122 and the memory access module 14 (DMA).

[0072] Specifically, during the execution of the steps and tasks involved in the above embodiments, the NPU chip:

[0073] The command distribution unit 11 is configured to write the configuration information of each subtask of the task to be reasoned into the computing unit 12.

[0074] The computing unit 12 is used to perform calculations on the current subtask based on the configuration information and by calling computing resources to obtain the subtask processing result.

[0075] The command distribution unit 11 is configured to allocate the unexecuted current sub-test task in the current sub-task to the idle computing resources if it is determined that there are idle computing resources in the computing unit 12 that are no longer processing the current sub-task.

[0076] The computing unit 12 is also used to call the idle computing resources to run the current sub-test task in order to perform system testing on the current sub-task.

[0077] The on-chip memory is used to temporarily store the results of completed subtask processing and subtask testing.

[0078] The memory access module 14 is also used to write the completed subtask processing results and subtask test results in the on-chip memory 13 into the off-chip memory 21 of the computer device.

[0079] refer to Figure 5 In the specific implementation, during the execution phase of the subtask Partition, the NPU first executes Partition1. During the execution of the first network layer Layer#1, Layer#1 will occupy specific computing resources of computing unit 12 for a certain period of time. In this example, the first network layer Layer#1 first simultaneously occupies the first computing resource 121 (MMP Core) and the second computing resource 122 (VSP Core). When the first network layer Layer#1 no longer occupies the second computing resource 122 (VSP Core), the command distribution unit distributes the first subtest task Test#1 to the second computing resource 122 (VSP Core) according to the pre-compiled instruction information to perform functional safety checks on it, thereby realizing the insertion execution of the first subtest task Test#1.

[0080] After Partition 1 is executed, the NPU executes Partition 2 sequentially. Similarly, the second network layer (Layer #2) will occupy the first computing resource 121 (MMP Core) and the second computing resource 122 (VSP Core) for a certain period. After Layer #2 finishes occupying the first computing resource 121 (MMP Core), the command dispatch unit distributes the second sub-test task (Test #2) to the first computing resource 121 (MMP Core) for testing, thus achieving the interleaved execution of the second sub-test task (Test #2). After Layer #2 and the second sub-test task (Test #2) are completed, the NPU continues to execute the remaining third network layer (Layer #3) of the sub-task Partition, and after completion, the current neural network model inference and STL testing tasks are terminated.

[0081] Secondly, refer to Figure 6 The task compilation method for the artificial intelligence model involved in the preset compilation method mentioned in the test task execution method embodiment mainly includes steps A1 to A3:

[0082] Step A1: Compile and divide the artificial intelligence model according to the number of network layers to obtain a first undetermined partitioning result. The first undetermined partitioning result includes L sub-task partitions, and each sub-task partition contains several consecutive network layers; where L is a positive integer.

[0083] Step A2: According to the first preset processing method, allocate several sub-test tasks in the self-test task set to L sub-task partitions in the first pending partition result to obtain the second pending partition result;

[0084] Step A3: Iterate through each sub-task partition of the second undetermined partitioning result to adjust the number of network layers and the number of sub-test tasks in each sub-task partition until the performance balance condition is met between each sub-task partition, and stop iterating to obtain a target partitioning result that meets the performance balance condition. The target partitioning result includes N sub-task partitions. Each sub-task partition in the target partitioning result is used as the basic scheduling unit of the NPU to complete the inference of the inference task related to the artificial intelligence model.

[0085] It should be noted that the execution entity of the task compilation method in this application embodiment is a compilation device equipped with application compilation software tools. This compilation device is deployed on the side of relevant R&D personnel (different from the computer device involved in the first aspect embodiment). The compilation method of steps A1 to A3 above is completed by the compilation device to obtain the program corresponding to the target partitioning result. The program corresponding to the target partitioning result is saved to the memory 21 of the computer device involved in the first aspect embodiment, and called by the system-on-a-chip (SoC) of the computer device in the first aspect embodiment to complete the steps of the test task execution method of the artificial intelligence model involved in the first aspect embodiment.

[0086] For example, the compilation device will call the AI ​​compiler to execute the compilation method of this embodiment to compile the inference task, and then perform inference on the inference task related to the artificial intelligence model based on the final compilation result. The performance balance condition can characterize that the computational cost of each subtask (hereinafter, for ease of description, each task is referred to as a Partition) is the same, or the computational cost of each subtask Partition is close. In this embodiment, a neural network model is used as an example of an artificial intelligence model for illustration.

[0087] Specifically, the compilation method proposed in this application can utilize the deterministic characteristics of neural network models to uniformly divide complex (artificial intelligence) network models into multiple computational sub-tasks with similar computational complexity. The determinism of neural network models can be understood as the network layers in each neural network model being statically fixed and unchanging, and the total number of network layers in each neural network model being pre-defined (maintaining the total number of network layers in each neural network model unchanged in the later stages).

[0088] Each subtask partition consists of several consecutive network layers and also serves as the basic scheduling unit of the NPU on the computer device involved in the first aspect embodiment. The NPU can sequentially execute these subtask partitions to run the application software normally. This allows the NPU to execute the application at the subtask partition level during the inference task of the neural network model. This facilitates the allocation of each subtest task in the self-test task set to a subtask partition. During the execution of each subtask partition, the NPU can synchronously execute the subtest tasks assigned to that partition, eliminating the need, as in conventional techniques, to execute all subtest tasks in the self-test task set only after completing a complete inference task.

[0089] First, for step A2, the AI ​​compiler needs to perform the first compilation and partitioning of the artificial intelligence model according to the number of network layers, resulting in L sub-task partitions. The L sub-task partitions obtained after the first compilation and partitioning are called the "first undetermined partitioning results".

[0090] Secondly, for step A2, refer to Figure 7 and Figure 8 The preset processing method in step A2 includes the following sub-steps a21 to a23:

[0091] Sub-step a21: Simulate the first undetermined partitioning result to obtain the simulation result;

[0092] Specifically, the subtask Partition is sent to the first simulation model for transaction-level simulation to simulate the behavior of the subtask Partition in each clock cycle and obtain simulation results. In this embodiment, the Cycle Accuracy Level Model (CA Model) can be used as the first simulation model. The CPU can first send the subtask Partition generated by the AI ​​compiler in sub-step A1 to the Cycle Accuracy Level Model (CA Model) for transaction-level simulation.

[0093] Sub-step a22 involves analyzing the scheduling performance metrics of each subtask partition in the simulation results to obtain the analysis results;

[0094] Specifically, the simulation results are fed into a performance analysis tool for analysis of the scheduling performance metrics of the simulation results, yielding analysis results. These scheduling performance metrics include at least information such as the runtime of each subtask partition and the NPU computing resource usage of each subtask partition.

[0095] Sub-step a23: Based on the analysis results, allocate several sub-test tasks in the self-test task set to L sub-task partitions in the first pending partitioning result to obtain the second pending partitioning result;

[0096] Specifically, the STL Scheduler accepts each STL sub-test task from the standard STL self-test task set, as well as the analysis results obtained in sub-step a22 above. Based on the different NPU computing resource occupancy of each sub-task partition, it automatically inserts an appropriate number of STL sub-test tasks into the corresponding sub-task partitions. This fully utilizes the idle NPU computing resources during partition execution, facilitating STL functional safety checks that run in parallel with the neural network inference task model. In this embodiment, the L sub-task partitions with STL sub-test tasks obtained in sub-step a23 are referred to as the "second pending partition result," also known as the back-annotation result. The STL Scheduler feeds back the back-annotation result to the AI ​​compiler.

[0097] Finally, for step A3, the AI ​​compiler will perform a new round of compilation, adjustment, simulation and evaluation on the second undetermined partition result (back-annotation result) until the optimal target partition result is obtained (i.e., the performance balance condition is met among the various sub-tasks). The CPU will then select the network model with the target partition result as a task to be processed and schedule it into the task queue.

[0098] In specific implementation, the principle of uniformly dividing a complex (artificial intelligence) network model into multiple computationally similar sub-tasks using the deterministic characteristics of the neural network model is as follows: For example, in the assembly stage, there are M artificial intelligence models (e.g., Model#1, Model#2, Model#3...Model#M). Taking the partitioning process of an artificial intelligence model Model#1 as an example, assume that Model#1 has a total of five network layers (Layer#1, Layer#2, Layer#3, Layer#4, Layer#5); and the self-test task set has two sub-test tasks (Test#1, Test#2).

[0099] The AI ​​model Model#1, containing five network layers, is input into the AI ​​editor. Two sub-test tasks, Test#1 and Test#2, from the test task set are input into the STL scheduler according to the task compilation method provided in this application. This aims to achieve performance balancing (i.e., automatically inserting appropriately sized STL sub-test tasks into corresponding sub-task partitions based on the different NPU computing resource occupancy of each sub-task partition, to fully utilize idle NPU computing resources during partition execution, achieving the goal of equal or similar computational load for each sub-task partition). Model #1 is divided into two partitions (Partition #1 and Partition #2), which are arranged sequentially according to the subtask order. Iterative processing is performed according to a preset processing method. When Partition #1 includes consecutive network layers Layer #1 and Layer #2, and is inserted with sub-test task Test #1, and Partition #2 includes consecutive network layers Layer #3, Layer #4, and Layer #5, and is also inserted with sub-test task Test #2, the performance balance condition is met among the subtask partitions of Model #1. The existing M artificial intelligence models can be compiled sequentially in the above manner, ultimately enabling each neural network model in this embodiment to receive NPU loading and scheduling at this moderately granular scheduling unit, and to perform STL sub-test tasks to self-test the system while executing the subtask partitions of each neural network model.

[0100] The beneficial effects of this application embodiment are as follows: the compilation device can pre-compile the tasks to be processed of the artificial intelligence model, so that it has multiple sub-task partitions that achieve performance balance, and each sub-test task in the self-test task set will be allocated to the corresponding sub-task partition as needed; in this way, when computer devices such as vehicle terminals are running target applications related to artificial intelligence, such as autonomous driving, during the execution of each sub-task partition by the NPU, the NPU can synchronously execute the sub-test tasks allocated to that sub-task partition, realizing efficient utilization of NPU resources and parallel execution of self-test function safety checks, effectively improving the response efficiency and resource utilization of the NPU.

[0101] refer to Figure 9 This application also proposes a test task execution device based on an artificial intelligence model, the execution device comprising:

[0102] The subtask execution module 901 is used to call the artificial intelligence model and traverse the N subtasks of the task to be reasoned in the order of the subtasks, and call m computing resources to process the network layer in each subtask to perform task reasoning on the task to be reasoned, where N and m are both positive integers.

[0103] The sub-test task execution module 902 is used to, during the process of calling the m computing resources to process the current network layer in the current sub-task, if it is determined that there are idle computing resources among the m computing resources that are no longer processing the current network layer, allocate the current sub-test task that has not been executed in the current sub-task to the idle computing resources, so as to perform system testing on the current sub-task.

[0104] Understandably, Figure 9 The corresponding task execution device can be an integrated circuit, installed in the computer device described in the above embodiments, to cooperate in executing the embodiments. Figure 2 and Figure 3 The test task execution methods involved.

[0105] It should be noted that the information interaction and execution process between the above-mentioned devices / modules / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0106] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0107] This application embodiment also provides a vehicle equipped with a computer device as described in the above embodiment. When the computer device runs the target application, the chip of the computer device executes the steps of the test task execution method of the artificial intelligence model and the task compilation method of the artificial intelligence model as described above.

[0108] This application also provides a network device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement the steps in any of the above method embodiments.

[0109] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0110] This application provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the above-described method embodiments.

[0111] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0112] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0113] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0114] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0115] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0116] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for executing a test task of an artificial intelligence model, characterized in that, The test task execution method is applied to a chip, which includes an NPU and is deployed on a computer device storing a target application and a set of self-test tasks. When the computer device runs the target application, the NPU is configured to invoke an artificial intelligence model to infer the inference task of the target application. The set of self-test tasks is used to perform system testing on the inference task and includes several sub-test tasks. The task to be inferred is divided into N subtasks according to a preset compilation method, and each sub-test task is assigned to a subtask according to the preset compilation method. The subtasks are arranged in subtask order, each subtask contains a network layer, and the network layers in the subtasks are arranged in network layer order. Each network layer belongs to one of the artificial intelligence models. The task execution method includes: The NPU invokes the artificial intelligence model and sequentially traverses the N subtasks of the task to be reasoned according to the order of the subtasks, and calls m computing resources to process the network layer in each subtask in order to perform task reasoning on the task to be reasoned, where N and m are both positive integers. During the process of the NPU calling the m computing resources to process the current network layer in the current subtask, if it is determined that there are idle computing resources among the m computing resources that are no longer processing the current network layer, the NPU will allocate the current sub-test task that has not been executed in the current subtask to the idle computing resources to perform system testing on the current subtask.

2. The method of claim 1, wherein, The performance balance condition among the N subtasks is at least characterized by the fact that the subtasks occupy the same or similar amount of computing resources among the m computing resources, and the computational load of each subtask is the same or similar. 3.A task compiling method of an artificial intelligence model, characterized in that, The compilation method includes: The inference tasks related to the artificial intelligence model are compiled and divided according to the number of network layers to obtain a first undetermined division result. The first undetermined division result includes L sub-tasks, and each sub-task contains several consecutive network layers; where L is a positive integer. According to the first preset processing method, several sub-test tasks in the self-test task set are assigned to L sub-tasks in the first undetermined partitioning result to obtain the second undetermined partitioning result; The process iterates over each subtask of the second undetermined partitioning result to adjust the number of network layers and the number of subtest tasks in each subtask until the performance balance condition is met between each subtask, and stops the iteration to obtain a target partitioning result that meets the performance balance condition. The target partitioning result includes N subtasks. Each subtask in the target partitioning result is used as the basic scheduling unit of the NPU to complete the inference of the task to be inferred.

4. The compiling method of claim 3, wherein, According to the first preset processing method, several sub-test tasks in the self-test task set are assigned to L sub-tasks in the first undetermined partitioning result to obtain the second undetermined partitioning result, including: The first undetermined partitioning result is simulated to obtain the simulation result; The CPU analyzes the scheduling performance metrics of each subtask in the simulation results to obtain the analysis results; Based on the analysis results, several sub-test tasks in the self-test task set are assigned to L sub-tasks in the first pending partitioning result to obtain the second pending partitioning result.

5. The compilation method as described in claim 4, characterized in that, The scheduling performance metrics include at least the runtime of each subtask and the computational resource usage of each subtask.

6. A chip, characterized in that, The chip includes a CPU and an NPU, and is deployed on a computer device storing a target application and a set of self-test tasks. The NPU is configured to invoke an artificial intelligence model to perform inference on the target application's inference tasks when the CPU is running the target application. The set of self-test tasks is used to perform system testing on the inference tasks, and the set of self-test tasks includes several sub-test tasks. The task to be inferred is divided into N subtasks according to a preset compilation method, and each subtest task is assigned to a subtask according to the preset compilation method. The subtasks are arranged in order, each subtask contains a network layer, the network layers of the subtasks are arranged in order, and each network layer belongs to one of the artificial intelligence models. The NPU is used to call the artificial intelligence model and sequentially traverse the N subtasks of the task to be reasoned according to the order of the subtasks, and call m computing resources to process the network layer in each subtask in order to perform task reasoning on the task to be reasoned. In the process of calling the m computing resources to process the current network layer in the current subtask, the NPU is also configured to, if it is determined that there are idle computing resources among the m computing resources that are no longer processing the current network layer, allocate the current sub-test task that has not been executed in the current subtask to the idle computing resources, so as to perform system testing on the current subtask.

7. The chip as described in claim 6, characterized in that, The NPU includes a command distribution unit and a computing unit; The command distribution unit is configured to write the configuration information of each subtask of the task to be reasoned into the computing unit; The computing unit is used to perform computational processing on the current subtask based on the configuration information and by calling computing resources to obtain the subtask processing result. The command distribution unit is configured to allocate the currently unexecuted sub-test task in the current sub-task to the idle computing resources if it is determined that there are idle computing resources in the computing unit that are no longer processing the current sub-task. The computing unit is also used to call upon the idle computing resources to run the current sub-test task in order to perform system testing on the current sub-task.

8. The chip as described in claim 7, characterized in that, The computing unit includes at least a first computing resource and a second computing resource; The command distribution unit is further configured to, when it is determined that the first computing resource is processing the current sub-task and the second computing resource is no longer processing the current sub-task, allocate the unexecuted current sub-test task in the current sub-task to the second computing resource; The computing unit is also used to call the second computing resource to run the current sub-test task in order to perform system testing on the current sub-task.

9. A test task execution device based on an artificial intelligence model, characterized in that, The device includes: The subtask execution module is used to call the artificial intelligence model and traverse the N subtasks of the task to be reasoned in the order of the subtasks, and call m computing resources to process the network layer in each subtask to perform task reasoning on the task to be reasoned, where N and m are both positive integers. The sub-test task execution module is used to, during the process of calling the m computing resources to process the current network layer in the current sub-task, if it is determined that there are idle computing resources among the m computing resources that are no longer processing the current network layer, allocate the current sub-test task that has not been executed in the current sub-task to the idle computing resources, so as to perform system testing on the current sub-task.

10. A computer device, characterized in that, The computer device includes a chip as described in any one of claims 6 to 8, a memory, and a test task execution program for an artificial intelligence model stored in the memory and executable on the chip, wherein when the test task execution program for the artificial intelligence model is executed by the chip, it implements the steps of the test task execution method for the artificial intelligence model as described in claim 1 or 2.

Images