Information acquisition method and analysis system

Abstract

The invention provides an information acquisition method and an analysis system, which can be applied to the technical field of computers. The information acquisition method is applied to the analysis system. The analysis system is used for analyzing the plurality of arithmetic units. The plurality of arithmetic units are to implement a first set of instructions. The first instruction set is obtained by compiling a computer program for implementing the operator. A first execution sequence exists among a plurality of instructions in the first instruction set, and each instruction is executed by one of the plurality of arithmetic units according to the first execution sequence. The information acquisition method comprises the steps that an analysis system monitors whether each arithmetic unit in a plurality of arithmetic units executes an instruction or not; and according to whether each arithmetic unit executes the instruction, outputting a time period when each arithmetic unit is in an idle state and a time period when each arithmetic unit is in a non-idle state. Wherein the idle state is used for indicating that no instruction is executed, and the non-idle state is used for indicating that the instruction is being executed. And the user can know the actual use condition of the hardware resources when the operator is executed through the output result.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an information acquisition method and analysis system. Background Technology

[0002] A neural network consists of multiple operators, and the performance of neural network training or inference primarily depends on the performance of these operators. Here, the performance of neural network training or inference refers to the time required for neural network training or inference, while the performance of an operator refers to the execution time of the operator. During the operation of a neural network on a computing device, the computing device rationally allocates its hardware resources (such as computing resources, storage resources, etc.) to the multiple operators that make up the neural network, in order to make full use of the allocated hardware resources to accelerate the execution process of the operators.

[0003] It is generally believed that the higher the utilization rate of hardware resources allocated to an operator, the better the operator's performance. Therefore, for operator developers, the performance of their developed operators can be measured by the operator's hardware resource utilization rate. Furthermore, operator developers can optimize operators based on this utilization rate. However, the operator's hardware resource utilization rate is a statistical indicator that only reflects the overall execution status of the operator, providing very limited information to operator developers. Therefore, optimizing operators based on their hardware resource utilization rate is inefficient. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides an information acquisition method and analysis system that can provide feedback to users on the actual usage of hardware resources when an operator is executed.

[0005] Firstly, an information acquisition method is provided, which is applied to an analysis system. The analysis system analyzes multiple arithmetic units. These multiple arithmetic units implement a first instruction set. The first instruction set is compiled from a computer program used to implement operators, and includes multiple instructions. The multiple instructions have a first execution order, and each instruction is executed by one of the multiple arithmetic units according to this first execution order. The method includes: the analysis system monitoring whether each arithmetic unit executes an instruction; and, based on whether each arithmetic unit executes an instruction, outputting the time periods during which each arithmetic unit is in an idle state and the time periods during which it is in a non-idle state. The idle state indicates that no instruction is being executed, and the non-idle state indicates that an instruction is being executed.

[0006] In the above scheme, during the implementation of the operator, i.e., the execution of instructions in the first instruction set, the execution status of the arithmetic unit is monitored. Based on this, the idle and non-idle periods of the arithmetic unit are determined and fed back to the user. This allows the user to clearly understand the actual hardware resource usage when the operator is executed, as the arithmetic unit switches between busy and idle states during operator implementation. Furthermore, this solution eliminates the need to repeatedly run the computer program; running the program only once (i.e., executing the first instruction set once) is sufficient to obtain the actual hardware resource usage when the operator is executed.

[0007] In some possible implementations, the above method further includes: the analysis system acquiring the start and end times of the first time period; and outputting the positions of the instructions corresponding to the start and end times in the second instruction set. The first time period is the period during which the first arithmetic unit is idle. The first arithmetic unit belongs to the aforementioned multiple arithmetic units. The second instruction set is compiled from a computer program used to implement the aforementioned operator. The first instruction set is a subset of the second instruction set.

[0008] In the above scheme, after determining the period when the first arithmetic unit is in an idle state, the user is provided with relevant instructions to put the first arithmetic unit into an idle state. It can be understood that the relevant instructions to put the first arithmetic unit into an idle state are equivalent to instructions to reduce the utilization rate of the first arithmetic unit, i.e., problem instructions. Therefore, the location of the problem instructions is fed back to the user, so that the user can find the part of the entire computer program that needs to be optimized in the development of the program.

[0009] In some possible implementations, the first instruction is the instruction corresponding to the start time or the instruction corresponding to the end time. The position of the first instruction in the second instruction set is indicated by the computer program corresponding to the first instruction, or by the position of the computer program corresponding to the first instruction in the computer program used to implement the operator.

[0010] In the above scheme, the location of the problem instruction is represented by the specific content or location of the computer program corresponding to the problem instruction. This provides users with information that they are familiar with, thereby improving the user experience.

[0011] In some possible implementations, the method further includes: the analysis system outputting the periods during which the second arithmetic unit is in an idle state and the periods during which it is in a non-idle state within the first time period. The second arithmetic unit belongs to the plurality of arithmetic units. The second arithmetic unit is used to execute a second instruction. The second instruction is the instruction between the instruction corresponding to the start time and the instruction corresponding to the end time, according to the first execution order.

[0012] The above solution not only provides users with the location of the problematic instruction but also the corresponding problem scenario: the execution status of instructions by other arithmetic units when the first arithmetic unit is idle. The problem scenario reflects which arithmetic units are working when the first arithmetic unit is idle; the specific details of the instructions executed by these units and the correlation between their execution; and information such as why the first arithmetic unit is not working while waiting for the processing results from other units. The information reflected in the problem scenario can guide users in optimizing their computer programs, improving their optimization efficiency. Furthermore, this technical solution only analyzes, records, and reports the problematic instruction and its corresponding problem scenario, without needing to record and report the execution status and scenario of each individual instruction. Therefore, this technical solution records and reports less information with higher quality, reducing the storage requirements of the analysis system and allowing users to directly obtain useful information to optimize their developing computer programs, thus improving their optimization efficiency.

[0013] In some possible implementations, the instructions executed by the first arithmetic unit are all computation instructions.

[0014] Since the execution of computation instructions is time-consuming, the usage time of the arithmetic unit that executes the computation instructions largely determines the execution time of the operator, that is, the performance of the operator. Therefore, this technical solution can focus on analyzing the actual usage of the arithmetic unit that executes the computation instructions, and guide users to optimize their developed computer programs in order to improve the utilization rate of the arithmetic unit that executes the computation instructions.

[0015] In some possible implementations, the aforementioned multiple processing units belong to the same processor.

[0016] In some possible implementations, the aforementioned multiple arithmetic units are used to implement a third instruction set. The third instruction set and the aforementioned first instruction set are obtained by compiling different computer programs for implementing the same operator. The method further includes: the analysis system monitoring whether each arithmetic unit executes instructions during the implementation of the third instruction set; based on whether each arithmetic unit executes instructions during the implementation of the third instruction set, outputting the periods during which each arithmetic unit is in an idle state and the periods during which it is in a non-idle state during the implementation of the third instruction set; and outputting the periods during which each arithmetic unit is in an idle state and the periods during which it is in a non-idle state during the implementation of the first instruction set.

[0017] In the above scheme, when running the user-optimized computer program, the user is provided with the execution instructions of each arithmetic unit before and after optimization, so that the user can compare the overall optimization effect of the computer program.

[0018] Secondly, an analysis system is provided for analyzing multiple arithmetic units. The multiple arithmetic units implement a first instruction set. The first instruction set is compiled from a computer program used to implement operators, and includes multiple instructions. The multiple instructions have a first execution order, and each instruction is executed by one of the multiple arithmetic units according to the first execution order. The analysis system includes a monitoring unit and an output unit. The monitoring unit monitors whether each arithmetic unit executes an instruction. The output unit outputs the periods when each arithmetic unit is in an idle state and the periods when it is in a non-idle state, based on whether each arithmetic unit executes an instruction. The idle state indicates that no instruction is being executed, and the non-idle state indicates that an instruction is being executed.

[0019] In some possible implementations, the aforementioned analysis system includes a determining unit. This determining unit is used to acquire the start and end times of the first time period. The output unit is also used to output the positions of the instructions corresponding to the start and end times in the second instruction set. The first time period is the period during which the first arithmetic unit is idle. The first arithmetic unit belongs to the aforementioned plurality of arithmetic units. The second instruction set is compiled from a computer program used to implement the aforementioned operators. The first instruction set is a subset of the second instruction set.

[0020] In some possible implementations, the first instruction is the instruction corresponding to the start time or the instruction corresponding to the end time. The position of the first instruction in the second instruction set is indicated by the computer program corresponding to the first instruction, or by the position of the computer program corresponding to the first instruction in the computer program used to implement the operator.

[0021] In some possible implementations, the output unit is further configured to output the periods during which the second arithmetic unit is idle and the periods during which it is not idle within the first time period. The second arithmetic unit is one of the aforementioned multiple arithmetic units. The second arithmetic unit is configured to execute a second instruction. The second instruction is the instruction between the instruction corresponding to the start time and the instruction corresponding to the end time, according to the first execution order.

[0022] In some possible implementations, the instructions executed by the first arithmetic unit are all computation instructions.

[0023] In some possible implementations, the aforementioned multiple processing units belong to the same processor.

[0024] In some possible implementations, the aforementioned multiple arithmetic units are used to implement a third instruction set. The third instruction set and the aforementioned first instruction set are obtained by compiling different computer programs for implementing the same operator.

[0025] The aforementioned monitoring unit is also used to monitor whether each arithmetic unit executes instructions during the implementation of the third instruction set.

[0026] The aforementioned output unit is also used to output the idle time period and the non-idle time period of each arithmetic unit during the implementation of the third instruction set, depending on whether each arithmetic unit executes an instruction during the implementation of the third instruction set, and to output the idle time period and the non-idle time period of each arithmetic unit during the implementation of the first instruction set.

[0027] Thirdly, an analysis device cluster is provided, including at least one analysis device, each analysis device including a processor and memory.

[0028] The processor of the at least one analysis device is configured to execute instructions stored in the memory of the at least one analysis device, so that the cluster of analysis devices performs the method as described in any of the first aspects.

[0029] Fourthly, a computer program product containing instructions is provided, which, when executed by an analysis device, cause the analysis device to perform the method as described in any of the first aspects.

[0030] Fifthly, a computer-readable storage medium is provided, characterized in that it includes computer program instructions, which, when executed by an analysis device, perform the method as described in any of the first aspects. Attached Figure Description

[0031] Figure 1A This is an architecture diagram of an analysis system provided in an embodiment of this application;

[0032] Figure 1B This is an architecture diagram of another analysis system provided in an embodiment of this application;

[0033] Figure 2 This is a flowchart illustrating an information acquisition method provided in an embodiment of this application;

[0034] Figure 3A This is a schematic diagram of a flow chart provided in an embodiment of this application;

[0035] Figure 3B This is a schematic diagram of another flow chart provided in an embodiment of this application;

[0036] Figure 3C This is a schematic diagram of another flow chart provided in an embodiment of this application;

[0037] Figure 3D This is a schematic diagram of another flow chart provided in an embodiment of this application;

[0038] Figure 3E This is a schematic diagram of another flow chart provided in an embodiment of this application;

[0039] Figure 4 This is a schematic diagram of the structure of an analytical device provided in this application;

[0040] Figure 5 This is a schematic diagram of the structure of an analysis device cluster provided in an embodiment of this application;

[0041] Figure 6 This is a schematic diagram of another analysis device cluster structure provided in an embodiment of this application. Detailed Implementation

[0042] The embodiments of this application are described below with reference to the accompanying drawings.

[0043] Before introducing the embodiments of this application, the application scenarios involved in this application will be explained.

[0044] Artificial intelligence (AI) models are mathematical or computational models used to simulate human intelligent behavior, including neural network models, deep learning models, genetic algorithm models, and so on.

[0045] Operators are used to represent the basic operations or functions in an AI model. An AI model typically consists of multiple operators. Operators define the computational process and learning rules of the AI ​​model, thus determining how the AI ​​model learns from input data and how it makes predictions or inferences. For example, neural network models include convolution operators, pooling operators, and fully connected operators. These operators define the structure and computational process of the neural network, such as the convolution and pooling operations in a convolutional neural network. Deep learning models include activation function operators (such as ReLU and Sigmoid), batch normalization operators, and Dropout operators. These operators are used to implement various functions and characteristics of deep learning models. Genetic algorithm models include selection operators, crossover operators, and mutation operators. These operators are used to simulate the biological evolution process to achieve population evolution and optimization.

[0046] Operator developers (hereinafter referred to as users) implement their developed operators by writing computer programs. Therefore, when a computing device runs a computer program, it is considered to be implementing an operator. Specifically, the computing device executes the computer program by executing a set of instructions compiled from the computer program. Taking a computing device containing an AI accelerator card as an example, the specific process of a computing device containing an AI accelerator card running a computer program used to implement an operator is as follows: the processor in the computing device controls the compiler in the computing device to compile the computer program into an instruction set containing multiple instructions. The processor then sends the instruction set to the AI ​​accelerator card. The controller (control unit) in the AI ​​accelerator card distributes the instructions in the instruction set to multiple AI processors in the AI ​​accelerator card. The multiple AI processors execute these instructions in parallel, thereby achieving high-performance processing and computation of the computer program.

[0047] The aforementioned computing devices are electronic devices used for computing, processing, and storing data, including servers, supercomputers, personal computers, workstations, mobile devices, and so on. AI accelerator cards are hardware devices specifically designed to accelerate AI model computation. They typically operate independently or work in conjunction with processors in computing devices (such as central processing units (CPUs), data processing units (DPUs), and graphics processing units (GPUs). AI processors, also known as minimal execution units, are the basic computing units in AI accelerator cards. AI processors (such as neural network processing units (NPUs), tensor processing units (TPUs), and machine learning units (MLUs)) are usually designed to execute instructions independently, and multiple AI processors can work in parallel to accelerate the execution efficiency of multiple instructions.

[0048] It should be understood that the above-described example of running a computer program to implement operators on a computing device containing an AI accelerator card is merely an example and is not intended to be specific. In practical applications, the computer program to implement operators can also run on a computing device containing an AI chip, and it can also run on a computing device without an AI accelerator card and AI chip. The AI ​​chip is a chip that integrates a dedicated AI processor (such as an NPU), typically integrated into mobile devices, IoT devices, or embedded systems, for real-time artificial intelligence inference and processing on the device side.

[0049] Through the above process of running a computer program to implement operators on a computing device containing an AI accelerator card, it can be seen that the instruction set used to implement operators can be implemented by a single AI processor or by multiple AI processors working together. When the instruction set is implemented by multiple AI processors, one of the AI ​​processors only executes a portion of the instructions in the instruction set.

[0050] Since an AI model consists of multiple operators, its training or inference performance primarily depends on the performance of these operators. Here, AI model training or inference performance refers to the time required for training or inference, while operator performance refers to the execution time of the operator. During the execution of an AI model on a computing device, the device rationally allocates its hardware resources (such as computing resources and storage resources) to the multiple operators that make up the AI ​​model, fully utilizing the allocated hardware resources to accelerate the operator execution process.

[0051] It is generally believed that the higher the utilization rate of hardware resources allocated to an operator, the better the operator's performance. Therefore, for users, the performance of their developed operators can be measured by the operator's hardware resource utilization rate. Furthermore, users can optimize operators based on this utilization rate. However, the operator's hardware resource utilization rate is a statistical indicator that only reflects the overall execution status of the operator, providing very limited information to the user. Therefore, optimizing operators based on their hardware resource utilization rate is inefficient.

[0052] To address the low efficiency of operator optimization based on hardware resource utilization, this application provides an analysis system applicable to computing devices implementing operators. Specifically, during operator implementation, the system monitors whether the arithmetic units in the computing device execute instructions. Based on the instruction execution status, it outputs to the user the periods when the arithmetic units are idle and non-idle. Through the output, the user can obtain the switching times of the arithmetic units from busy to idle and from idle to busy during operator implementation, thus clearly understanding the actual hardware resource usage when the operator is executed.

[0053] See Figure 1A , Figure 1A This is an architecture diagram of an analysis system provided in an embodiment of this application. For example... Figure 1A As shown, the architecture includes a client 100, a computing system 200, and an analysis system 300. Communication between the client 100 and the computing system 200, between the computing system 200 and the analysis system 300, and between the client 100 and the analysis system 300 can be achieved via wired or wireless means.

[0054] The client 100 is used to obtain the computer program input by the user for implementing the operator, and send the computer program to the computing system 200 for processing.

[0055] The computing system 200 is used to run computer programs, thereby implementing operators. Specifically, multiple arithmetic units in the computing system 200 execute multiple instructions obtained by compiling computer programs.

[0056] The analysis system 300 is used to obtain information about the instruction execution status of the arithmetic units in the computing system 200, analyze the instruction execution status of the arithmetic units to obtain analysis results, and send the analysis results to the client 100. Specifically, the analysis system 300 monitors whether the arithmetic units are executing instructions, and then sends the client 100 the periods when the arithmetic units are in an idle state and the periods when they are in a non-idle state, based on the instruction execution status of the arithmetic units. The idle state indicates that no instructions are being executed, and the non-idle state indicates that instructions are being executed.

[0057] In some possible implementations, the client 100, computing system 200, and analysis system 300 can all be deployed on computing devices. The client 100, computing system 200, and analysis system 300 can be deployed on different computing devices, or on the same computing device. When the client 100, computing system 200, and analysis system 300 are deployed on different computing devices, at least two of them can be deployed on different computing devices within the same computing device cluster, or on different computing devices within different computing clusters. The computing device cluster can include multiple of the aforementioned computing devices. The specific deployment can be determined based on the actual application scenario, and this application does not impose specific limitations.

[0058] In one specific implementation, the client 100 can also be deployed on a terminal device. The terminal device is an electronic device used to access computing devices, including personal computers, smartphones, handheld devices, tablets, mobile laptops, all-in-one handheld consoles, smart conferencing devices, smart advertising devices, and so on.

[0059] In some possible implementations, the computing units in the computing system 200 are used to train or infer the AI ​​model.

[0060] The arithmetic units in the computing system 200 are the hardware units within the computing device where the computing system 200 resides. These include various types such as the arithmetic and logic unit (ALU), floating-point unit (FPU), vector processing unit (VPU), and storage units. The ALU performs arithmetic and logical operations, including addition, subtraction, multiplication, division, logical AND, and logical OR operations. The FPU performs floating-point operations, including floating-point addition, subtraction, multiplication, division, and square root operations. The VPU performs vectorized computations, including matrix multiplication and convolution operations. Storage units store data and intermediate results, including register files and caches.

[0061] Optionally, the computing system 200 may include hardware devices or chips (such as AI accelerator cards, AI chips, etc.) to accelerate AI model computation. In this case, the computing unit in the computing system 200 includes not only the hardware units in the computing device, but also the hardware units in the hardware devices or chips used to accelerate AI model computation.

[0062] Taking an AI accelerator card as an example of a hardware device used to accelerate AI model computation, the computing unit includes the computing unit of the AI ​​processor within the AI ​​accelerator card. The AI ​​processor's computing unit includes not only ALU and FPU, but also memory transfer engine (MTE) units, cube units, vector units, and so on. The MTE unit is used for data transfer and management between the computing device's global memory and the AI ​​processor's on-chip cache, including data loading, storage, and movement operations. Optionally, the MTE unit includes a first MTE unit and a second MTE unit. The first MTE unit is used to transfer data from global memory to the on-chip cache. The second MTE unit is used to transfer data from the on-chip cache to global memory. The cube unit is used to perform computational operations on the data, such as matrix multiplication and convolution operations. The vector unit is used to perform vector operations on the data, such as applying activation functions like ReLU, Sigmoid, and Tanh to the AI ​​model's output data.

[0063] For ease of explanation, the following text will use AI accelerator cards as examples of hardware devices or chips that accelerate AI model calculations.

[0064] In some possible implementations, the analysis system 300 includes multiple units. For example, see... Figure 1B , Figure 1B This is an architecture diagram of another analysis system provided in the embodiments of this application.

[0065] exist Figure 1B In the analysis system 300, there are monitoring unit 310, determination unit 320 and output unit 330.

[0066] The monitoring unit 310, the determining unit 320, and the output unit 330 can be implemented in software or hardware. For example, the implementation of the monitoring unit 310 will be described below. Similarly, the implementation of the determining unit 320 and the output unit 330 can refer to the implementation of the monitoring unit 310.

[0067] As an example of a software functional unit, monitoring unit 310 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, or a container. Further, the aforementioned computing instance may be one or more. For example, monitoring unit 310 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0068] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0069] As an example of a hardware functional unit, the monitoring unit 310 may include at least one computing device, such as a server. Alternatively, the monitoring unit 310 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The aforementioned PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), a data processing unit (DPU), a neural network processing unit (NPU), a system-on-chip (SoC), an offload card, an accelerator card, or any combination thereof.

[0070] The monitoring unit 310 includes multiple computing devices that can be distributed in the same region or in different regions. Similarly, the monitoring unit 310 can be distributed within the same Availability Zone (AZ) or in different AZs. Likewise, the monitoring unit 310 can be distributed within the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, GALs, DPUs, NPUs, SoCs, offloading cards, and accelerator cards.

[0071] The functions of each unit in the analysis system 300 are explained below.

[0072] The monitoring unit 310 is used to monitor whether each of the multiple arithmetic units in the computing system 200 is executing instructions. This process can be seen in the following... Figure 2 The execution process of step S101.

[0073] The determining unit 320 is used to determine the idle period and the non-idle period of each arithmetic unit based on whether each arithmetic unit is executing instructions. The idle state indicates that no instructions are being executed, and the non-idle state indicates that instructions are being executed. This process can be seen in the following... Figure 2 The execution process of step S102.

[0074] Output unit 330 is used to send the idle time periods and non-idle time periods of each processing unit to client 100. This process can be seen below. Figure 2 The execution process of step S103.

[0075] It should be understood that the units in the aforementioned analysis system 300 are based on... Figure 1B The analysis system 300 is described using the monitoring unit 310 to the output unit 330 as an example. In practical applications, the analysis system 300 may include more or fewer units, and this application does not make any specific limitations.

[0076] It should be understood that the above Figure 1A or Figure 1B The architecture of the analysis system is illustrated using the example of one client 100 and one computing system 200 communicating with the analysis system 300. In practical applications, the number of client 100 and computing system 200 can be one or more, and this application does not make a specific limitation.

[0077] In summary, the analysis system 300 provided in this application can provide feedback to the user on the switching times of each operation unit from busy to idle and from idle to busy during the execution of the computer program used to implement the operator, so that the user can clearly know the actual usage of hardware resources when the operator is executed.

[0078] The above text combined Figure 1A as well as Figure 1B The analysis system 300 provided in the embodiments of this application has been introduced. Next, a flowchart of an information acquisition method provided in the embodiments of this application will be presented.

[0079] See Figure 2 , Figure 2 This is a flowchart illustrating an information acquisition method provided in an embodiment of this application. The information acquisition method provided in this application is applied to the above... Figure 1A or Figure 1B The analysis system architecture includes analysis system 300. For example... Figure 2 As shown in the embodiments of this application, the information acquisition method includes:

[0080] S101: The monitoring unit in the analysis system monitors whether each of the multiple arithmetic units executes instructions.

[0081] The analysis system can be Figure 1B The analysis system 300 in the middle, the monitoring unit can be Figure 1B The monitoring unit 310 in the analysis system 300. The range of computing units monitored by the monitoring unit can be configured as needed, as detailed in application scenarios 1 and 2 below:

[0082] Application Scenario 1: The monitoring unit monitors multiple arithmetic units to implement the first instruction set.

[0083] Multiple processing units belong to the computing system. The computing system can be... Figure 1B The computing system 200 in the middle.

[0084] The first instruction set is compiled from a computer program used to implement the operator. The first instruction set includes multiple instructions. The instructions in the first instruction set have a first execution order, and each instruction is executed by one of a plurality of arithmetic units according to this first execution order. In a possible application scenario, the computer program used to compile the first instruction set is a part of the entire computer program used to implement the operator. Therefore, for ease of distinction, the instruction set compiled from the entire computer program used to implement the operator can be called the second instruction set, and the first instruction set is a subset of the second instruction set.

[0085] In application scenario 1, if the computing system does not have an AI accelerator card installed, the multiple processing units used to implement the first instruction set are all processing units within the same processor in the computing device, such as ALU, FPU, VPU, etc. If the computing system has an AI accelerator card installed, the multiple processing units are all processing units within the same AI processor in the AI ​​accelerator card, such as the first MTE unit, the second MTE unit, the Cube unit, etc. The computing system can be... Figure 1B The computing system 200 in the example. For ease of explanation, the following content regarding application scenario 1 will be explained using the example of multiple computing units belonging to the same AI processor.

[0086] In some possible implementations, the instructions in the first instruction set have different types, such as data load instructions, store instructions, computation instructions, etc. Specifically, data load instructions instruct the loading of data from the computing device's global memory into the AI ​​processor's on-chip cache. Store instructions instruct the storage of data from the AI ​​processor's on-chip cache into the computing device's global memory. Computation instructions instruct the execution of computational operations, such as matrix multiplication, convolution operations, activation functions, etc.

[0087] Typically, the instructions executed by a single arithmetic unit are all of the same type. Continuing with the example of multiple arithmetic units in the same AI processor, the first MTE unit executes data loading instructions, the second MTE unit executes storage instructions, and the Cube unit executes computation instructions.

[0088] In application scenario 1, multiple arithmetic units execute instructions from a first instruction set in a first execution order. Each instruction is executed by one of the multiple arithmetic units. Continuing with the example of a first MTE unit, a second MTE unit, and a Cube unit within the same AI processor, since computational operations involve data, the execution of a computational instruction by the Cube unit often requires the cooperation of the first and second MTE units. First, the first MTE unit executes a data loading instruction, loading the data to be computed from the computing device's global memory into the AI ​​processor's on-chip cache. After the first MTE unit completes the data loading instruction, the Cube unit executes the computation instruction. Specifically, the Cube unit reads the data to be computed from the on-chip cache, performs computation on the data, obtains the computation result, and writes the result into the AI ​​processor's on-chip cache. After the Cube unit completes the computation instruction, the second MTE unit executes a storage instruction, storing the computation result from the AI ​​processor's on-chip cache into the computing device's global memory.

[0089] It can be assumed that data loading instructions, calculation instructions, and storage instructions generally exist in pairs and have a relatively fixed execution order. When the first instruction set includes multiple calculation instructions that need to be executed by the Cube unit, the Cube unit can only execute these calculation instructions one by one. Once the execution order of these calculation instructions on the Cube unit is determined, since the data loading instructions and storage instructions that accompany each calculation instruction already have a relatively fixed execution order with the calculation instructions, the execution order of all instructions in the first instruction set can be obtained, which is the first execution order.

[0090] Application Scenario 2: The monitoring unit monitors multiple arithmetic units to implement the second instruction set.

[0091] The second instruction set is compiled from the entire computer program used to implement the operator.

[0092] In application scenario 2, if the computing system does not have an AI accelerator card installed, the multiple computing units belong to different processors within the computing device. If the computing system has an AI accelerator card installed, the multiple computing units belong to different AI processors within the AI ​​accelerator card. Specifically, the controller in the AI ​​accelerator card distributes the instructions in the second instruction set to the multiple AI processors within the AI ​​accelerator card. These AI processors then execute the instructions in the second instruction set in parallel, thereby implementing the operator. Therefore, each AI processor receives a portion of the instructions in the second instruction set. It can be considered that each of these AI processors receives the first instruction set, and each AI processor is used to implement the first instruction set. All the computing units in these AI processors together constitute the multiple computing units in application scenario 2 used to implement the second instruction set. Furthermore, the multiple computing units in application scenario 2 include the multiple computing units in application scenario 1.

[0093] In some possible implementations, the instructions in the second instruction set are of different types, such as data loading instructions, storage instructions, computation instructions, and so on.

[0094] In application scenario 2, different AI processors independently execute the first instruction set assigned to them. The process of the first AI processor executing the first instruction set assigned to it does not interfere with the process of the second AI processor executing the first instruction set assigned to it. This means that the time period for the first AI processor to execute the first instruction set may or may not overlap with the time period for the second AI processor to execute the first instruction set.

[0095] In some possible implementations, the monitoring period of the monitoring unit is related to the application scenario.

[0096] Corresponding to application scenario 1 of step S101, since multiple arithmetic units are used to implement the first instruction set, the monitoring unit can monitor whether each arithmetic unit executes instructions in real time during the implementation of the first instruction set, and it can also monitor whether each arithmetic unit executes instructions in real time during the implementation of the second instruction set, or it can monitor whether the arithmetic unit executes instructions continuously from the beginning of each arithmetic unit's operation. Specifically, the process of implementing the first instruction set refers to the process from the execution of the first instruction in the first instruction set to the execution of the last instruction in the first instruction set. Similarly, the process of implementing the second instruction set refers to the process from the execution of the first instruction in the second instruction set to the execution of the last instruction in the second instruction set.

[0097] Corresponding to application scenario 2 of step S101, since multiple arithmetic units are used to implement the second instruction set, the monitoring unit can either monitor whether each arithmetic unit executes instructions in real time during the implementation of the second instruction set, or monitor whether the arithmetic unit executes instructions at the beginning of each arithmetic unit's operation.

[0098] In some possible implementations, the monitoring unit communicates with each arithmetic unit through a debugging interface or driver to obtain the execution status of each arithmetic unit in real time, thereby achieving real-time monitoring of whether each arithmetic unit is executing instructions. Real-time monitoring typically refers to continuously monitoring changes in a certain indicator or state at very short time intervals. Typically, real-time monitoring can be performed at the microsecond level or even shorter time intervals. Real-time monitoring of whether an arithmetic unit is executing instructions means continuously monitoring the execution status of the arithmetic unit at very short time intervals. Execution status refers to whether the arithmetic unit is executing instructions. If the arithmetic unit is not executing instructions, the execution status is idle; if the arithmetic unit is executing instructions, the execution status is non-idle.

[0099] Specifically, the computing unit typically provides a debugging interface or a communication interface.

[0100] When the monitoring unit is connected to the debugging interface of the arithmetic unit, the monitoring unit can obtain the execution status of the arithmetic unit in real time through the debugging interface. This process can be referenced to the process of obtaining the CPU execution status using software performance analysis tools (such as AMD CodeXL) or hardware performance analyzers (such as JTAG Debugger). For the sake of brevity, it will not be elaborated here.

[0101] When the monitoring unit connects to the communication interface of the computing unit, the monitoring unit can send commands to the computing unit through the driver program and receive the execution status returned by the computing unit based on the commands. In this scenario, the monitoring unit sends commands to the computing unit in real time, thus obtaining the real-time execution status of the computing unit.

[0102] In one specific implementation, the monitoring unit can also obtain instruction execution information from the arithmetic unit in real time through a debugging interface or a communication interface. This instruction execution information refers to the specific details of the instructions being executed by the arithmetic unit, including the instruction type, execution stage, and execution cycle.

[0103] S102: The determination unit in the analysis system determines the idle time period and the non-idle time period of each arithmetic unit based on whether each arithmetic unit executes an instruction.

[0104] The determining unit can be Figure 1B The determination unit 320 in the analysis system 300.

[0105] By analyzing whether each arithmetic unit executes instructions in real time through the system's monitoring unit, the monitoring results for each arithmetic unit during the monitoring period can be obtained. Based on the monitoring results of an arithmetic unit, the determination unit can determine the periods during which that arithmetic unit is executing instructions and the periods during which it is not executing instructions. Since the execution state corresponding to an arithmetic unit executing instructions is a non-idle state, and the execution state corresponding to an arithmetic unit not executing instructions is an idle state, the periods during which the arithmetic unit is in a non-idle state and the periods during which it is in an idle state can be determined.

[0106] In application scenario 1 of step S101 above, if the monitoring period is the process of implementing the first instruction set, and the multiple computing units are the first MTE unit, the second MTE unit, and the Cube unit in the same AI processor, then the determining unit can obtain the time periods when the first MTE unit, the second MTE unit, and the Cube unit are in a non-idle state and the time periods when they are in an idle state, respectively, based on the monitoring results of the first MTE unit, the second MTE unit, and the Cube unit.

[0107] For example, the monitoring period is 0-10 (unit: microseconds / μs), and the monitoring results of the first MTE unit, the second MTE unit, and the Cube unit are shown in Table 1 below. For the sake of brevity, Table 1 uses "0" to indicate that no instruction was executed and "1" to indicate that an instruction was being executed.

[0108] Table 1

[0109] Time period (μs) First MTE Unit Second MTE unit Cube unit 0-2 1 0 0 2-4 0 0 1 4-5 0 1 0 5-6.5 1 0 0 6.5-8.5 0 0 1 8.5-10 0 1 0

[0110] As shown in Table 1:

[0111] The first MTE unit is in a non-idle state during the periods of 0-2 and 5-6.5, and is in an idle state during the remaining periods.

[0112] The second MTE unit is in a non-idle state during the periods of 4-5 and 8.5-10, and is in an idle state during the remaining periods.

[0113] The Cube cells are in a non-idle state during the periods of 2-4 and 6.5-8.5, and are idle during the remaining periods.

[0114] It should be understood that the monitoring period and monitoring results in the example of application scenario 1 above are merely examples, and this application does not impose any specific limitations.

[0115] In application scenario 2 of step S101 above, if the monitoring period is the process of implementing the second instruction set, and the multiple computing units include computing units in multiple AI processors, then the determining unit can also obtain the time period when each computing unit is in a non-idle state and the time period when it is in an idle state based on the monitoring results of each computing unit.

[0116] For example, the monitoring period is 0-30 (unit: microseconds / μs), and the multiple computing units include the computing units in the first AI processor and the second AI processor, specifically including the first MTE unit, the second MTE unit, and the Cube unit in the first AI processor, and the first MTE unit, the second MTE unit, and the Cube unit in the second AI processor.

[0117] In this context, the first MTE unit, second MTE unit, and Cube unit in the first AI processor can respectively correspond to the first MTE unit, second MTE unit, and Cube unit in the same AI processor in the example of application scenario 1 above. The monitoring results of the first MTE unit, second MTE unit, and Cube unit in the first AI processor are shown in Table 2 below; the monitoring results of the first MTE unit, second MTE unit, and Cube unit in the second AI processor are shown in Table 3 below. For the sake of brevity, in both Tables 2 and 3, "0" indicates that no instruction has been executed; and "1" indicates that an instruction is being executed.

[0118] Table 2

[0119]

[0120] Table 3

[0121]

[0122]

[0123] As shown in Table 2, in the first AI processor,

[0124] The first MTE unit is in a non-idle state during the periods of 0-2 and 5-6.5.

[0125] The second MTE unit is in a non-idle state during the periods of 4-5 and 6.5-10.

[0126] The periods during which the Cube cell is not idle are: 2-4, 6.5-8.5;

[0127] As shown in Table 3, in the second AI processor,

[0128] The first MTE unit is in a non-idle state during the periods of 5-8 and 14-18.

[0129] The second MTE unit is in a non-idle state during the following periods: 12-14 and 28-30.

[0130] The Cube cells are in a non-idle state during the following periods: 8-12 and 18-28.

[0131] During the monitoring period, the time during which each computing unit is not in a non-idle state is considered the time during which it is in an idle state.

[0132] It should be understood that the monitoring period and monitoring results in the above examples are merely examples and are not specifically limited in this application.

[0133] S103: The output unit in the analysis system sends the time periods when each processing unit is in an idle state and the time periods when it is in a non-idle state to the client.

[0134] The output unit can be Figure 1B The output unit 330 in the analysis system 300.

[0135] In some possible implementations, there are multiple ways for the output unit to output the time periods when the arithmetic unit is in different states. For details, please refer to the following output methods 1-3:

[0136] Output Method 1: The output unit summarizes the idle and non-idle periods of each processing unit and sends it to the client in tabular form. Correspondingly, the client receives the execution status table. Each entry in the execution status table represents the idle and non-idle periods of a processing unit within the monitoring period.

[0137] For example of application scenario 1 corresponding to step S102 above, the execution status table can be Table 1 in step S102 above.

[0138] In the example of application scenario 2 corresponding to step S102 above, the execution status table includes Table 2 and Table 3 in step S102 above, for example, it can be a merged table of Table 2 and Table 3.

[0139] Output Method 2: The output unit records the periods when each processing unit is idle and the periods when it is not idle, and sends these records to the client in the form of a document. The client then receives the execution status document accordingly.

[0140] Corresponding to the example of application scenario 1 in step S102 above, the specific content of the execution status document is as follows:

[0141] "The first MTE unit is in a non-idle state during the periods of 0-2 and 5-6.5, and is in an idle state during the periods of 2-5 and 6.5-10;

[0142] The second MTE unit is in a non-idle state during the periods of 4-5 and 8.5-10, and is in an idle state during the periods of 0-4 and 6.5-8.5.

[0143] The Cube cells are in a non-idle state during the following periods: 2-4 and 6.5-8.5; and in an idle state during the following periods: 0-2, 4-6.5, and 8.5-10.

[0144] For example of application scenario 2 corresponding to step S102 above, the specific content of the execution status document includes the first MTE unit, the second MTE unit, and the Cube unit in the first AI processor, the time period during which the first MTE unit, the second MTE unit, and the Cube unit in the second AI processor are in an idle state within 0-30μs, and the time period during which they are in a non-idle state.

[0145] Output Method 3: The output unit draws the idle time periods and non-idle time periods of each processing unit and sends them to the client as images. The client then receives the pipeline diagram accordingly.

[0146] Corresponding to the example of application scenario 1 in step S102 above, the flow chart is as follows: Figure 3A As shown. See also Figure 3A , Figure 3A This is a schematic diagram of a flow chart provided in an embodiment of this application.

[0147] Corresponding to the example of application scenario 2 in step S102 above, the flow chart is as follows: Figure 3B As shown. See also Figure 3B , Figure 3B This is a schematic diagram of another flow chart provided in an embodiment of this application.

[0148] Combination Figure 3A and Figure 3B As can be seen, using a pipeline diagram can visually show the switching points of the computing unit from busy to idle and from idle to busy.

[0149] It should be understood that the above-described output methods 1-3 for different states of the arithmetic unit are merely examples. In practical applications, other methods can also be used to output the different states of the arithmetic unit. This application does not impose any specific limitations.

[0150] In summary, during the implementation of the operator, i.e., the execution of instructions in the first instruction set, the execution status of the arithmetic unit is monitored. Based on this, the idle and non-idle periods of the arithmetic unit are determined and fed back to the user. This allows the user to clearly understand the actual hardware resource usage during operator execution by observing the transition times from busy to idle and from idle to busy during the operator implementation process. Furthermore, this technical solution eliminates the need to repeatedly run the computer program; running the program only once (i.e., executing the first instruction set once) is sufficient to obtain the actual hardware resource usage during operator execution.

[0151] In some possible implementations, the determining unit in step S102 above obtains the start time and end time of the first time period, and determines the position of the instruction corresponding to the start time and end time in the second instruction set; then the output unit in step S103 above sends the position of the instruction corresponding to the start time and end time in the second instruction set to the client.

[0152] The first time period is the period during which the first processing unit is in an idle state. The first processing unit belongs to multiple processing units.

[0153] Continuing with the example of application scenario 1 in step S101 above, the first processing unit can be the first MTE unit, the second MTE unit, or the Cube unit within the same AI processor. Taking the Cube unit as an example of the first processing unit, in the example of application scenario 1 in step S102 above, the first time period can be 0-2, 4-6.5, or 8.5-10. Taking 4-6.5 as an example of the first time period, the start time of the first time period is 4 and the end time is 6.5.

[0154] Optionally, the instructions executed by the first arithmetic unit are all calculation instructions.

[0155] Optionally, the first time period is defined as the time period during which the first processing unit is in an idle state and the duration is greater than or equal to a specified duration. The specified duration is determined by the user.

[0156] The method for determining the location of the instructions corresponding to the start and end times of the first time period in the second instruction set can be either automatically determined by the analysis system or determined by the analysis system in conjunction with user-provided dot-mapping commands. Dot-mapping commands are used to instruct the output of preset information (such as specific strings, variable values, or the path of computer program execution).

[0157] The implementation method automatically determined by the analysis system is as follows:

[0158] If the instruction corresponding to the start time or the instruction corresponding to the end time is taken as the first instruction, and the computer program corresponding to the first instruction is determined, then the position of the first instruction in the second instruction set can be represented by either the specific content of the computer program corresponding to the first instruction or by the position of the computer program corresponding to the first instruction in the entire computer program used to implement the operator (such as the line number of the computer program code).

[0159] When the location of the first instruction in the second instruction set is represented by the specific content of the computer program corresponding to the first instruction, the output unit sends the specific content of the computer program corresponding to the instruction at the start time and the specific content of the computer program corresponding to the instruction at the end time to the client. The user can then search for the computer program corresponding to the instruction at the start time and the computer program corresponding to the instruction at the end time within the entire computer program they developed to implement the operator. The computer program between these two computer programs can then be optimized to improve the operator's performance.

[0160] When the position of the first instruction in the second instruction set is represented by the position of the computer program corresponding to the first instruction within the entire computer program used to implement the operator (e.g., the line number of the computer program code), the output unit sends the line number of the computer program corresponding to the instruction at the start time (e.g., line 10) and the line number of the computer program corresponding to the instruction at the end time (e.g., line 22) to the client. The user can then search for the computer program between lines 10 and 22 within the entire computer program they developed to implement the operator as the program to be optimized, thereby improving the performance of the operator.

[0161] The implementation method determined by the analysis system in conjunction with the user-provided point-marking commands is as follows:

[0162] After obtaining the start and end times of the first time period through the client, the user searches for the computer program to be confirmed from the entire computer program used to implement the operator, based on the start and end times. Then, the user inserts dot commands before and after the computer program to be confirmed, that is, inserts two dot commands.

[0163] The user sends the entire computer program containing two dot commands to the computing system via a client. The computing system then compiles the computer program containing the two dot commands to obtain a new second instruction set. Multiple arithmetic units within the computing system that implemented the original second instruction set then implement the new second instruction set.

[0164] During the implementation of the new second instruction set, the analysis system monitors whether the instructions compiled from dotted commands are being executed. When it detects that an instruction compiled from one of the dotted commands has started execution, the analysis system records the start time as the first time point; when it detects that an instruction compiled from another dotted command has started execution, the analysis system records the start time as the second time point. The analysis system then sends the first and second time points to the client.

[0165] Optionally, to facilitate user observation, the analysis system can send the first and second time points combined with a pipeline diagram as output to the user. Specifically, on the pipeline diagram used to record the idle and non-idle periods of each arithmetic unit implementing the original or new second instruction set, the first and second time points are marked, as well as the start and end times of the first time period. In the case where the first arithmetic unit is a Cube unit, and the start time of the first time period is 4 and the end time is 6.5, the marking result is as follows: Figure 3C As shown. See also Figure 3C , Figure 3C This is a schematic diagram of another flow chart provided in an embodiment of this application.

[0166] After obtaining the first and second moments through the client, users can compare the first and second moments with the start and end moments of the first time period to determine whether the computer program to be confirmed is the computer program to be optimized.

[0167] If the first moment is close to the start of the first time period and the second moment is close to the end of the first time period, it indicates that the computer program to be confirmed is the computer program to be optimized. Then, the position of the computer program to be confirmed in the entire computer program used to implement the operator is the position of the instruction corresponding to the start and end times of the first time period in the second instruction set.

[0168] If the first moment is not close to the start of the first time period, or the second moment is not close to the end of the first time period, it indicates that the computer program to be confirmed is not the computer program to be optimized. In this case, the user needs to search for new computer programs to be confirmed from the entire computer program and use the analysis system again to determine whether the new computer program to be confirmed is the computer program to be optimized, until the user finds the computer program to be optimized.

[0169] In summary, after determining the period during which the first arithmetic unit is idle, the user is provided with relevant instructions to put the first arithmetic unit into an idle state. It can be understood that the relevant instructions to put the first arithmetic unit into an idle state are equivalent to instructions that reduce the utilization rate of the first arithmetic unit, i.e., problem instructions. Therefore, the location of the problem instructions is fed back to the user, so that the user can find the parts of the entire computer program that need to be optimized in the development of the program.

[0170] In addition, using the specific content or location of the computer program corresponding to the problem instruction to represent the location of the problem instruction can provide users with information they are familiar with, thereby improving the user experience.

[0171] In scenarios where the instructions executed by the first arithmetic unit are all computation instructions, the execution time of the arithmetic unit that executes the computation instructions is long, so the usage time of the arithmetic unit that executes the computation instructions largely determines the execution time of the operator, that is, the performance of the operator. Therefore, this technical solution can focus on analyzing the actual usage of the arithmetic unit that executes the computation instructions, and guide users to optimize their developed computer programs in order to improve the utilization rate of the arithmetic unit that executes the computation instructions.

[0172] For scenarios where the duration of the first time period is greater than or equal to the specified duration, this technical solution can focus on analyzing the duration of all idle time periods of the first computing unit. This can guide users to focus on optimizing their developed computer programs in a way that can significantly improve the utilization rate of the computing unit.

[0173] In some possible implementations, the output unit sends to the client the periods during which the second processing unit is idle and the periods during which it is not idle within the first time period.

[0174] The second arithmetic unit is one of multiple arithmetic units used to execute the second instruction. The second instruction is the instruction between the instruction corresponding to the start time and the instruction corresponding to the end time. When the multiple arithmetic units to which the second arithmetic unit belongs are used to implement the first instruction set, the second instruction is the instruction between the instruction corresponding to the start time and the instruction corresponding to the end time, in accordance with the first execution order.

[0175] The following is combined with Figure 3D Examples are provided. See also Figure 3D , Figure 3D This is a schematic diagram of another flow chart provided in an embodiment of this application.

[0176] exist Figure 3D In this process, the first processing unit is the Cube unit, and the first time period starts at time 4 and ends at time 6.5.

[0177] Since between the completion of a calculation instruction in Cube unit 4 (i.e., the instruction corresponding to the start time of the first time period) and the start of another calculation instruction in 6.5 (i.e., the instruction corresponding to the end time of the first time period), a storage instruction is first executed by the second MTE unit, and then a data loading instruction is executed by the first MTE unit. Therefore, both the storage instruction and the data loading instruction are second instructions, and both the first MTE unit and the second MTE unit are second operation units.

[0178] Therefore, the output unit needs to send the following to the client: the idle time period and the non-idle time period of the first MTE unit between 4 and 6.5; and the idle time period and the non-idle time period of the second MTE unit between 4 and 6.5. If a pipeline diagram is used to send the data to the client, the sending result is shown below. Figure 3D The flowchart is shown in the dashed box.

[0179] It should be understood that Figure 3D The pipeline diagram within the dashed box that needs to be sent to the client is determined when the first processing unit is a Cube unit and the start time of the first time period is 4 and the end time is 6.5. If the first processing unit is another processing unit, or the first time period is another time period, Figure 3D The specific content of the pipeline diagram that needs to be sent to the client within the dashed box will change. In practical applications, Figure 3D The pipeline diagram that needs to be sent to the client can be determined according to user needs or actual application scenarios, and this application does not impose specific limitations.

[0180] In summary, this technical solution not only provides users with the location of the problematic instruction but also the corresponding problem scenario: the execution status of instructions by other arithmetic units when the first arithmetic unit is idle. The problem scenario reflects which arithmetic units are working when the first arithmetic unit is idle; the specific details of the instructions executed by these units and the correlation between their execution; and information such as why the first arithmetic unit is not working while waiting for the processing results from other arithmetic units. The information reflected in the problem scenario can guide users in optimizing their computer programs, improving their optimization efficiency. Furthermore, this technical solution only analyzes, records, and reports the problematic instruction and its corresponding problem scenario, without needing to record and report the execution status and scenario of each individual instruction. Therefore, this technical solution records and reports less information with higher quality, reducing the storage requirements of the analysis system and allowing users to directly obtain useful information to optimize their developing computer programs, thus improving their optimization efficiency.

[0181] In this embodiment, to distinguish the arithmetic units in different application scenarios (including application scenario 1 and application scenario 2) in step S101, each arithmetic unit in application scenario 2 that implements the second instruction set can be referred to as the third arithmetic unit. Therefore, in application scenario 2, the analysis system implements: monitoring whether the third arithmetic unit executes instructions, and determining and outputting the time periods when the third arithmetic unit is in an idle state and the time periods when it is in a non-idle state based on whether the third arithmetic unit executes instructions. The specific execution process can be found in the descriptions of application scenario 2 in steps S101, S102, and S103 above, and will not be repeated here.

[0182] In some potential application scenarios, users optimize their developed computer programs for implementing operators based on feedback from the analysis system, obtaining optimized computer programs, which are then recorded on the client side. The client then sends the optimized computer program to the computing system. The computing system compiles the optimized computer program to obtain a third instruction set. Multiple arithmetic units in the computing system that implement the first instruction set further implement the third instruction set. Since the first instruction set is obtained by compiling the unoptimized computer program, and the third instruction set is obtained by compiling the optimized computer program, the first and third instruction sets are different instruction sets. The process of multiple arithmetic units implementing the third instruction set is similar to the process of multiple arithmetic units implementing the first instruction set in application scenario 1 of step S101 above, and will not be repeated here.

[0183] During the implementation of the third instruction set, the analysis system monitors whether each of the multiple arithmetic units executes instructions. Based on whether each arithmetic unit executes instructions, it outputs the idle time period and the non-idle time period of each arithmetic unit during the implementation of the third instruction set. It also outputs the idle time period and the non-idle time period of each arithmetic unit during the implementation of the first instruction set. This process can be referred to in the relevant descriptions of application scenario 1 in steps S101, S102, and S103 above, and will not be repeated here.

[0184] In this application scenario, if the results are output in the form of a flow chart, the output result will be as follows: Figure 3E As shown. See also Figure 3E , Figure 3E This is a schematic diagram of another flow chart provided in an embodiment of this application. Users can observe... Figure 3E This allows us to understand whether the optimized computer program significantly improves the utilization of each computing unit compared to the original program.

[0185] It should be noted that, in application scenario 2 of step S101 above, the analysis system can also output the idle time period and the non-idle time period of each arithmetic unit in the process of implementing the instruction set compiled from the optimized computer program, and output the idle time period and the non-idle time period of each arithmetic unit in the process of implementing the instruction set (i.e., the second instruction set) compiled from the unoptimized computer program.

[0186] In summary, when running a user-optimized computer program, the system provides the user with information on the execution instructions of each arithmetic unit before and after optimization, allowing the user to compare the overall optimization effect of the computer program.

[0187] See Figure 4 , Figure 4 This is a schematic diagram of the structure of an analytical device provided in an embodiment of this application. For example... Figure 4 As shown, the analysis device 400 provided in this application includes: a bus 401, a processor 402, a memory 403, and a communication interface 404. The processor 402, the memory 403, and the communication interface 404 communicate with each other via the bus 401. The analysis device 400 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the analysis device 400.

[0188] Bus 401 can be a Peripheral Component Interconnect Express (PCIe) bus, an Extended Industry Standard Architecture (EISA) bus, a Unified Bus (Ubus or UB), a Compute Express Link (CXL), a Cache Coherent Interconnect for Accelerators (CCIX), etc. The Unified Bus is also known as the Lingqu Bus. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 4 The bus 401 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 401 may include a path for transmitting information between various components of the analysis device 400 (e.g., memory 403, processor 402, communication interface 404).

[0189] Processor 402 may include any one or more of the following: central processing unit (CPU), graphics processing unit (GPU), microprocessor (MP), digital signal processor (DSP), ASIC, FPGA, CPLD, NPU, SoC, offload card, accelerator card, etc.

[0190] Memory 403 may include volatile memory, such as random access memory (RAM). Processor 402 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD). Furthermore, memory 403 may also be implemented using storage class memory (SCM), phase change memory (PCM), or other types of storage media.

[0191] It is worth noting that the same type of storage medium can be configured in the same analysis device to realize the function of memory 403, or two or more types of storage media can be configured to realize the function of memory 403. This application does not limit this.

[0192] The memory 403 stores executable program code, and the processor 402 executes the executable program code to implement the aforementioned functions respectively. Figure 1B The analysis system 300 performs the functions of the monitoring unit 310 and the output unit 330, thereby executing the aforementioned functions respectively. Figure 2 The steps in the information acquisition method are executed by the monitoring unit and output unit in the analysis system.

[0193] Optionally, the program code in memory 403 used to implement the functions of monitoring unit 310 and output unit 330 is the same as described above. Figure 1B The relationship between the monitoring unit 310, the determination unit 320, and the output unit 330 in the analysis system 300 can also be dynamically adjusted according to business needs, so that the program code stored in the memory 403 can also achieve the aforementioned Figure 1BThe analysis system 300 includes the functions of one or more units among the monitoring unit 310, the determination unit 320, and the output unit 330. For example, the program code in the memory 403 used to implement the function of the output unit 330 can not only achieve the aforementioned functions... Figure 1B The output unit 330 in the analysis system 300 can also perform the functions described above. Figure 1B The function of the determination unit 320 in the analysis system 300.

[0194] In summary, memory 403 stores information for performing the aforementioned tasks. Figure 2 The instructions for obtaining information are as follows. For the sake of brevity, they will not be elaborated here.

[0195] The communication interface 404 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the analysis device 400 and other analysis devices or communication networks.

[0196] As one possible implementation, the analysis device 400 may also include a chip system, which includes a processor and a power supply circuit. The power supply circuit supplies power to the processor, and the processor performs the aforementioned operations. Figure 2 The information acquisition method involves operational steps performed by the analysis system. Optionally, the processor can implement the aforementioned steps by executing program code stored in memory 403 within the analysis device 400. Figure 1B The analysis system 300 includes the functions of one or more units among the monitoring unit 310, the determination unit 320, and the output unit 330, thereby achieving the aforementioned functions. Figure 2 The information acquisition method involves operational steps executed by the analysis system. These steps can be implemented using a CPU, or via devices such as a GPU, DPU, NPU, XPU, SoC, offload card, accelerator card, or AI chip.

[0197] As one possible implementation, the analysis device 400 may include multiple types of processors 402, that is, the analysis device 400 is a heterogeneous device. For example, the analysis device 400 includes a CPU and a GPU, and the aforementioned functions can be performed by at least one of the processors 402. Figure 2 The information acquisition method involves operational steps performed by the analysis system. For the sake of brevity, these steps will not be elaborated upon here.

[0198] See Figure 5 , Figure 5 This is a schematic diagram of an analysis device cluster provided in an embodiment of this application. The analysis device cluster provided in this application includes at least one analysis device. The analysis device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the analysis device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0199] like Figure 5 As shown, the analysis device cluster includes at least one analysis device 400. The memory 403 of one or more analysis devices 400 in the analysis device cluster may store the same memory for performing the aforementioned operations. Figure 2 The instructions for the steps executed by the analysis system in the information acquisition method.

[0200] In some possible implementations, the memory 403 of one or more analysis devices 400 in the analysis device cluster may also store memory for performing the aforementioned operations. Figure 2 The information acquisition method involves a portion of the instructions executed by the analysis system. In other words, a combination of one or more analysis devices 400 can jointly execute the instructions for performing the aforementioned steps. Figure 2 The instructions for the steps executed by the analysis system in the information acquisition method.

[0201] It should be noted that the memory 403 in different analysis devices 400 within the analysis device cluster can store different instructions, each used to execute the aforementioned instructions. Figure 1B The analysis system 300 contains some of the functions. That is, the instructions stored in the memory 403 of different analysis devices 400 can realize the functions of one or more units in the monitoring unit 310 and the output unit 330.

[0202] Optionally, the program code in memory 403 used to implement the functions of monitoring unit 310 and output unit 330 is the same as described above. Figure 1B The relationship between the monitoring unit 310, the determination unit 320, and the output unit 330 in the analysis system 300 can also be dynamically adjusted according to business needs, so that the program code stored in the memory 403 can also achieve the aforementioned Figure 1B The analysis system 300 includes the functions of one or more units among the monitoring unit 310, the determination unit 320, and the output unit 330.

[0203] See Figure 6 , Figure 6 This is a schematic diagram of another analytical device cluster structure provided in an embodiment of this application. In some possible implementations, one or more analytical devices in the analytical device cluster can be connected via a network. The network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 6 As shown, the two analysis devices 400A and 400B are connected via a network. Specifically, they are connected to the network through the communication interface in each analysis device. In this possible implementation, the memory 403 in analysis device 400A stores instructions for performing the functions of monitoring unit 310. The memory 403 in analysis device 400B stores instructions for performing the functions of output unit 330.

[0204] Figure 6 The connection method between the cluster of analysis devices shown can be such that, considering the information acquisition method provided in this application requires a large number of analysis results to be output, the function implemented by the output unit 330 is delegated to the analysis device 400B for execution.

[0205] It should be understood that Figure 6 The functions of the analytical device 400A shown can also be performed by multiple analytical devices 400. Similarly, the functions of the analytical device 400B can also be performed by multiple analytical devices 400.

[0206] This application also provides another analytical device cluster. The connection relationships between the analytical devices in this cluster can be similarly referenced. Figure 5 and Figure 6 The connection method of the analysis device cluster. The difference is that the memory 403 of one or more analysis devices 400 in this analysis device cluster can store the same information used to execute the aforementioned... Figure 2 The instructions for the steps executed by the analysis system in the information acquisition method.

[0207] In some possible implementations, the memory 403 of one or more analysis devices 400 in the analysis device cluster may also store memory for performing the aforementioned operations. Figure 2 The information acquisition method involves a portion of the instructions executed by the analysis system. In other words, a combination of one or more analysis devices 400 can jointly execute instructions for performing... Figure 2 The instructions for the steps executed by the analysis system in the information acquisition method.

[0208] It should be noted that the memory 403 in different analysis devices 400 within the analysis device cluster can store different instructions, each used to execute the aforementioned instructions. Figure 1B The analysis system 300 contains some of the functions. That is, the instructions stored in the memory 403 of different analysis devices 400 can realize the functions of one or more units in the monitoring unit 310 and the output unit 330.

[0209] Optionally, the program code in memory 403 used to implement the functions of monitoring unit 310 and output unit 330 is the same as described above. Figure 1B The relationship between the monitoring unit 310, the determination unit 320, and the output unit 330 in the analysis system 300 can also be dynamically adjusted according to business needs, so that the program code stored in the memory 403 can also achieve the aforementioned Figure 1B The analysis system 300 includes the functions of one or more units among the monitoring unit 310, the determination unit 320, and the output unit 330.

[0210] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on an analysis device or stored on any usable medium. When the computer program product is run on at least one analysis device, it causes the at least one analysis device to perform the aforementioned... Figure 2 The steps performed by the analysis system in the information acquisition method.

[0211] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any usable medium that the analysis device can store, or a data storage device such as a data center containing one or more usable media. The usable medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that direct the analysis device to perform the aforementioned actions. Figure 2 The steps performed by the analysis system in the information acquisition method.

[0212] It should be understood that in the embodiments of this application, "when," "...when," and "if" all refer to the device making corresponding processing under certain objective circumstances, and are not time-limited, nor do they require the device to make a judgment action, nor do they imply any other limitations.

[0213] It should be understood that in the embodiments of this application, "simultaneous" does not necessarily require that they occur at the same time, minute, second, or even at the same moment. When the times of occurrence are slightly different, they can also be understood as occurring "simultaneously".

[0214] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. An information acquisition method characterized by comprising: The method is applied to an analysis system for analyzing a plurality of operation units for implementing a first instruction set compiled from a computer program for implementing an operator, the first instruction set comprising a plurality of instructions having a first execution order, each instruction being executed by one of the plurality of operation units according to the first execution order, the method comprising: The analysis system monitors whether each operation unit of the plurality of operation units executes an instruction; The analysis system outputs a period of time when each operation unit is in an idle state and a period of time when each operation unit is in a non-idle state according to whether each operation unit executes an instruction, wherein the idle state indicates that no instruction is executed, and the non-idle state indicates that an instruction is executed.

2. The method of claim 1, wherein, The method further comprises: The analysis system obtains a start time and an end time of a first period of time, wherein the first period of time is a period of time when a first operation unit is in the idle state, and the first operation unit belongs to the plurality of operation units; The analysis system outputs positions of instructions corresponding to the start time and the end time in a second instruction set, wherein the second instruction set is compiled from the computer program for implementing the operator, and the first instruction set is a subset of the second instruction set.

3. The method of claim 2, wherein, A first instruction is the instruction corresponding to the start time or the instruction corresponding to the end time, and the position of the first instruction in the second instruction set is indicated by the computer program corresponding to the first instruction or by the position of the computer program corresponding to the first instruction in the computer program for implementing the operator.

4. The method according to claim 2 or 3, characterized in that, The method further comprises: The analysis system outputs a period of time when a second operation unit is in the idle state and a period of time when the second operation unit is in the non-idle state within the first period of time, wherein the second operation unit belongs to the plurality of operation units, and the second operation unit is used to execute a second instruction between the instruction corresponding to the start time and the instruction corresponding to the end time according to the first execution order.

5. The method according to any of claims 2-4, characterized by, The instructions executed by the first operation unit are all computing instructions.

6. The method according to any one of claims 1 to 5, characterized in that, The plurality of operation units belong to the same processor.

7. The method according to any of claims 1 to 6, characterized in that The plurality of operation units are used to implement a third instruction set, and the third instruction set and the first instruction set are obtained by compiling different computer programs for implementing the same operator, and the method further comprises: The analysis system monitors whether each operation unit executes an instruction in the process of implementing the third instruction set; The analysis system outputs a period of time when each operation unit is in the idle state and a period of time when each operation unit is in the non-idle state in the process of implementing the third instruction set according to whether each operation unit executes an instruction in the process of implementing the third instruction set, and outputs a period of time when each operation unit is in the idle state and a period of time when each operation unit is in the non-idle state in the process of implementing the first instruction set.

8. An analytical system, characterized by The analysis system is used for analyzing a plurality of operation units, the plurality of operation units are used for implementing a first instruction set, the first instruction set is compiled from a computer program used for implementing an operator, the first instruction set comprises a plurality of instructions, the plurality of instructions have a first execution order, each instruction is executed by one of the plurality of operation units according to the first execution order, the analysis system comprises an acquisition unit and an output unit, The acquisition unit is used for monitoring whether each operation unit of the plurality of operation units executes an instruction; The output unit is used for outputting a period in which each operation unit is in an idle state and a period in which each operation unit is in a non-idle state according to whether each operation unit executes an instruction, wherein the idle state is used for indicating that no instruction is executed, and the non-idle state is used for indicating that an instruction is executed.

9. The system of claim 8, wherein, The analysis system comprises a determination unit, The determination unit is used for acquiring a start time and an end time of a first period, wherein the first period is a period in which a first operation unit is in the idle state, and the first operation unit belongs to the plurality of operation units; The output unit is further used for outputting positions of instructions corresponding to the start time and the end time in a second instruction set, wherein the second instruction set is compiled from the computer program used for implementing the operator, and the first instruction set is a subset of the second instruction set.

10. The system of claim 9, wherein, A first instruction is the instruction corresponding to the start time or the instruction corresponding to the end time, and the position of the first instruction in the second instruction set is indicated by the computer program corresponding to the first instruction or by the position of the computer program corresponding to the first instruction in the computer program used for implementing the operator.

11. The system of claim 9 or 10, wherein, The output unit is further used for outputting a period in which a second operation unit is in the idle state and a period in which the second operation unit is in the non-idle state in the first period, wherein the second operation unit belongs to the plurality of operation units, and the second operation unit is used for executing a second instruction, and the second instruction is an instruction between the instruction corresponding to the start time and the instruction corresponding to the end time according to the first execution order.

12. The system of any of claims 9-11, wherein, The instructions executed by the first operation unit are all computing instructions.

13. The system of any of claims 8-12, wherein, The plurality of operation units belong to a same processor.

14. The system of any of claims 8-13, wherein, The plurality of operation units are used for implementing a third instruction set, and the third instruction set and the first instruction set are obtained by compiling different computer programs used for implementing a same operator, The acquisition unit is further used for monitoring whether each operation unit executes an instruction in a process of implementing the third instruction set; The output unit is further used for outputting a period in which each operation unit is in the idle state and a period in which each operation unit is in the non-idle state in the process of implementing the third instruction set according to whether each operation unit executes an instruction in the process of implementing the third instruction set, and outputting a period in which each operation unit is in the idle state and a period in which each operation unit is in the non-idle state in the process of implementing the first instruction set.

15. A cluster of analysis devices, characterized in that comprising at least one analysis device, each analysis device comprising a processor and a memory; the processor of the at least one analysis device is configured to execute instructions stored in the memory of the at least one analysis device to cause the cluster of analysis devices to perform the method of any one of claims 1 to 7.

16. A computer program product comprising instructions, characterized in that, the instructions, when executed by an analysis device, cause the analysis device to perform the method of any one of claims 1 to 7.

17. A computer-readable storage medium, characterized in that, computer program instructions, which, when executed by an analysis device, cause the analysis device to perform the method of any one of claims 1 to 7.

Images