Artificial intelligence chip test method, device and system

Abstract

The invention discloses an artificial intelligence chip test method, device and system, and relates to the technical field of artificial intelligence, and the method comprises the steps: a test host sends a first test instruction and at least one piece of first test data to an artificial intelligence chip, the first test instruction is used for indicating the artificial intelligence chip to execute target task reasoning based on the target artificial intelligence model; the test host records the sending time of the first test data as the inference starting time of the first test data, responds to an inference ending signal of the first test data returned by the artificial intelligence chip, and determines the sending time of the inference ending signal as the inference ending time of the first test data; and determining effective computing power information of the artificial intelligence chip based on the reasoning starting time and the reasoning ending time of each piece of first test data. According to the scheme, the effective computing power condition of the artificial intelligence chip can be accurately determined.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an artificial intelligence chip testing method, device and system. Background Art

[0002] Artificial Intelligence (AI) chips, also known as AI accelerators, are chips specifically designed to process AI-related tasks.

[0003] With the continuous development of AI technology, the computing power of AI models continues to increase. However, the computing power of AI models is limited by the computing and storage capabilities of the AI ​​chip that runs the AI ​​model. Generally speaking, when an AI chip runs an AI model, the effective computing power of the AI ​​chip is much lower than the nominal computing power of the AI ​​chip. Based on this, how to accurately determine the effective computing power that an AI chip can actually achieve when running an AI model (i.e., the so-called computing power of an AI chip) is a technical problem that technicians in this field need to solve. Summary of the invention

[0004] In view of the above problems, the present application provides an artificial intelligence chip testing method, device and system, which can more accurately determine the effective computing power of the artificial intelligence chip.

[0005] On the one hand, the present application provides an artificial intelligence chip testing method, which is applied to a test host, wherein a communication connection is established between the test host and the artificial intelligence chip to be tested, and at least one artificial intelligence model is deployed in the artificial intelligence chip, and the method includes:

[0006] Sending a first test instruction to the artificial intelligence chip, wherein the first test instruction is used to instruct the artificial intelligence chip to perform target task reasoning based on a target artificial intelligence model, where the target artificial intelligence model belongs to at least one artificial intelligence model deployed in the artificial intelligence chip;

[0007] Sending at least one first test data to the artificial intelligence chip;

[0008] Recording the sending time of the first test data as the inference start time of the first test data;

[0009] In response to an inference end signal of the first test data returned by the artificial intelligence chip, determining the issuance time of the inference end signal as the inference end time of the first test data, wherein the inference end signal is generated when the artificial intelligence chip completes the target task inference of the first test data using the target artificial intelligence model;

[0010] Based on the inference start time and the inference end time of each first test data, the effective computing power information of the artificial intelligence chip is determined.

[0011] In a possible implementation, determining the effective computing power information of the artificial intelligence chip based on the inference start time and the inference end time of each first test data includes:

[0012] For each first test data, determining the inference duration of the first test data based on the inference start time and the inference end time of the first test data;

[0013] Determine, based on the inference duration of each first test data, a benchmark inference duration required for the artificial intelligence chip to complete the inference of the target task;

[0014] Based on the benchmark inference time and the multiplication and addition operations of the target artificial intelligence model, the effective computing power index value of the artificial intelligence chip is determined.

[0015] In another possible implementation, determining the benchmark inference time required for the artificial intelligence chip to complete the target task inference based on the inference time of each first test data includes:

[0016] According to the order of the inference time of each first test data from small to large, the inference time at the Nth percentile is determined as the benchmark inference time required for the artificial intelligence chip to complete the target task inference, where N is an integer not less than 90 and less than 100;

[0017] The determining of the effective computing power index value of the artificial intelligence chip based on the benchmark inference time and the multiplication and addition operation amount of the target artificial intelligence model includes:

[0018] Determine the reciprocal of the benchmark inference duration as the equivalent frame rate;

[0019] Determining the computational amount of the target artificial intelligence model based on the multiplication and addition computational amount of the target artificial intelligence model;

[0020] Based on the product of the computational complexity of the target artificial intelligence model and the equivalent frame rate, the effective computing power index value of the artificial intelligence chip is determined.

[0021] In yet another possible implementation, sending at least one first test data to the artificial intelligence chip includes:

[0022] Determine the data transmission frame rate configured for the artificial intelligence chip, the set amount of test data to be sent each time, the target number of test data to be selected, and the total target number of test data required;

[0023] Determine the target number of first test data from the test data set;

[0024] Determining the set number of first test data to be sent currently according to the sequence of the target number of first test data;

[0025] When it is determined that the current moment reaches the data transmission moment according to the data transmission frame rate, sending the currently determined set number of first test data to the artificial intelligence chip;

[0026] If the cumulative total number of first test data sent to the artificial intelligence chip has not reached the target total number, return to execute the operation of determining the set number of first test data currently to be sent until the cumulative total number is not less than the target total number.

[0027] In yet another possible implementation, before sending the first test instruction to the artificial intelligence chip, the method further includes:

[0028] Determining a test mode for the artificial intelligence chip to be tested;

[0029] The sending a first test instruction to the artificial intelligence chip includes:

[0030] If the test mode is an inference computing power test, sending a first test instruction to the artificial intelligence chip;

[0031] The artificial intelligence chip testing method further includes:

[0032] If the test mode is an inference accuracy test, a second test instruction is sent to the artificial intelligence chip, wherein the second test instruction is used to instruct the artificial intelligence chip to call the target artificial intelligence model to perform target task inference on the test data set and determine the inference accuracy, wherein the test data set includes at least one second test data and the true value of the task result annotated by the second test data;

[0033] Obtain an inference accuracy result returned by the artificial intelligence chip, wherein the inference accuracy result is determined by the artificial intelligence chip based on the task result true value of each second test data and the inference task result, and the inference task result is obtained by the artificial intelligence chip performing the target task inference on the second test data based on the target artificial intelligence model.

[0034] In another possible implementation, before obtaining the reasoning accuracy result returned by the artificial intelligence chip, the method further includes:

[0035] If the test data set is not deployed in the artificial intelligence chip, each second test data in the test data set is sent to the artificial intelligence chip.

[0036] In another possible implementation, before determining the effective computing power information of the artificial intelligence chip, the method further includes:

[0037] Obtaining power values ​​reported by the artificial intelligence chip at different time points;

[0038] While or after determining the effective computing power information of the artificial intelligence chip, the method further includes:

[0039] Based on the power values ​​reported by the artificial intelligence chip at different time points, the average power and peak power of the artificial intelligence chip during the operation of the target artificial intelligence model are determined.

[0040] In another aspect, the present application further provides an artificial intelligence chip testing device, which is applied to a test host, wherein a communication connection is established between the test host and the artificial intelligence chip to be tested, wherein at least one artificial intelligence model is deployed in the artificial intelligence chip, and the device comprises:

[0041] A first instruction sending unit, configured to send a first test instruction to the artificial intelligence chip, wherein the first test instruction is used to instruct the artificial intelligence chip to perform target task reasoning based on a target artificial intelligence model, wherein the target artificial intelligence model belongs to at least one artificial intelligence model deployed in the artificial intelligence chip;

[0042] A data sending unit, configured to send at least one first test data to the artificial intelligence chip;

[0043] a time recording unit, configured to record the sending time of the first test data as the inference starting time of the first test data;

[0044] a time determination unit, configured to determine, in response to an inference completion signal of the first test data returned by the artificial intelligence chip, a time at which the inference completion signal is issued as an inference completion time of the first test data, wherein the inference completion signal is generated when the artificial intelligence chip completes the target task inference of the first test data using the target artificial intelligence model;

[0045] The computing power determination unit is used to determine the effective computing power information of the artificial intelligence chip based on the inference start time and the inference end time of each first test data.

[0046] In yet another aspect, the present application further provides an artificial intelligence chip testing system, comprising: a testing host and an artificial intelligence chip to be tested, wherein at least one artificial intelligence model is deployed in the artificial intelligence chip;

[0047] The test host comprises: a control module, a data sending module and a timing monitoring module with communication connection;

[0048] The control module, data transmission module and timing monitoring module are all in communication connection with the artificial intelligence chip;

[0049] The control module is used to send a first test instruction to the artificial intelligence chip, wherein the first test instruction is used to instruct the artificial intelligence chip to perform target task reasoning based on a target artificial intelligence model, and the target artificial intelligence model belongs to at least one artificial intelligence model deployed in the artificial intelligence chip;

[0050] The data sending module is used to send at least one first test data to the artificial intelligence chip under the control of the control module;

[0051] The artificial intelligence chip is used to respond to the first test instruction, call the target artificial intelligence model to perform the target task reasoning on the first test data; when the target task reasoning on the first test data is completed, send a reasoning end signal to the timing monitoring module;

[0052] The timing monitoring module is configured to detect that the data sending module sends the first test data to the artificial intelligence chip, record the sending time of the first test data as the inference start time of the first test data; and, in response to an inference end signal of the first test data returned by the artificial intelligence chip, determine the sending time of the inference end signal as the inference end time of the first test data;

[0053] The control module is also used to determine the effective computing power information of the artificial intelligence chip based on the inference start time and the inference end time of each first test data recorded by the timing monitoring module.

[0054] In a possible implementation, the control module is further used to determine the test mode to be tested of the artificial intelligence chip before sending the first test instruction to the artificial intelligence chip; if the test mode is an inference accuracy test, send a second test instruction to the artificial intelligence chip, the second test instruction is used to instruct the artificial intelligence chip to call the target artificial intelligence model to perform target task inference on the test data set and determine the inference accuracy, the test data set includes at least one second test data, and the task result true value marked by the second test data; obtain the inference accuracy result returned by the artificial intelligence chip;

[0055] The artificial intelligence chip is further used to, in response to the second test instruction, call the target artificial intelligence model to perform the target task reasoning on the second test data to obtain the reasoning task result of the second test data; and determine the reasoning accuracy result based on the task result true value of each second test data and the reasoning task result;

[0056] When the control module sends the first test instruction to the artificial intelligence chip, it is specifically used to send the first test instruction to the artificial intelligence chip if the test mode is an inference computing power test.

[0057] From the above content, it can be seen that the test host can instruct the artificial intelligence chip to call the target artificial intelligence model to perform the specified target task reasoning by sending the first test instruction to the artificial intelligence chip. On this basis, the test host will send the first test data to the artificial intelligence chip, and record the sending time of the first test data as the reasoning start time of the first test data. Moreover, after the artificial intelligence chip completes the target task reasoning of the first test data and returns the reasoning end signal, the test host will determine the sending time of the reasoning end signal as the reasoning end time of the first test data. Since the reasoning start time and reasoning end time of the first test data reflect the actual time taken by the artificial intelligence chip to actually run the artificial intelligence model to process the first test data, therefore, combined with the reasoning start time and reasoning end time of each first test data, the actual effective computing power of the artificial intelligence chip can be more accurately determined. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0059] Figure 1 A schematic diagram of a flow chart of the artificial intelligence chip testing method provided in this application;

[0060] Figure 2 A schematic diagram of a process of sending first test data from a test host to an artificial intelligence chip in the present application;

[0061] Figure 3 A schematic diagram of the composition architecture of the artificial intelligence chip testing system provided in this application;

[0062] Figure 4 Another flowchart of the artificial intelligence chip testing method provided in this application;

[0063] Figure 5A schematic diagram of the composition structure of the artificial intelligence chip testing device provided in this application. DETAILED DESCRIPTION

[0064] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation mode of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application. It is known to those skilled in the art that with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0065] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0066] The present application scheme is applicable to a test host to perform performance testing on an artificial intelligence chip to be tested, wherein the performance test at least includes a computing power test of the artificial intelligence chip.

[0067] In the present application, a communication connection is established between the test host and the artificial intelligence chip to be tested.

[0068] Among them, there can be multiple possibilities for the communication connection between the test host and the artificial intelligence chip, and there can be one or more communication leveling connections between the test host and the artificial intelligence chip, without any restriction.

[0069] For example, the communication connection between the test host and the artificial intelligence chip may include, but is not limited to: serial port communication based on one or more of Universal Asynchronous Receiver / Transmitter (UART), Mobile Industry Processor Interface (MIPI), High Definition Multimedia Interface (HDMI), Gigabit Multimedia Serial Links (GMSL), general-purpose input / output (GPIO), Controller Area Network (CAN) and Serial Peripheral Interface (SPI); the communication connection may also be an Ethernet-based communication connection.

[0070] In this application, the artificial intelligence chip to be tested can be any chip that can run an artificial intelligence (AI) model. For example, artificial intelligence chips include: tensor processing unit (TPU) chip, neural network processing unit (NPU) chip, graphics processing unit (GPU) chip, brain processing unit (BPU) and deep learning processing unit (DPU) and other AI processor chips.

[0071] The artificial intelligence chip can be a chip installed in any device. For example, the target device equipped with the artificial intelligence chip can be a terminal, a vehicle-mounted terminal or a server, etc., without any restriction.

[0072] Among them, at least one artificial intelligence model is deployed in the artificial intelligence chip. The artificial intelligence model deployed in the artificial intelligence chip can be deployed according to actual needs without restriction. For example, the artificial intelligence models deployed in the artificial intelligence chip include but are not limited to: Resnet18, Resnet50, Yolo, SSD, VGG and other AI algorithm models.

[0073] In one possible implementation, a test program may be deployed in an artificial intelligence chip, and the test program includes at least one artificial intelligence model. Of course, the test program may also be configured with software codes for performing different reasoning tasks based on each artificial intelligence model.

[0074] It is understandable that in actual applications, before testing the artificial intelligence chip, it is necessary to power the artificial intelligence chip and install various interface drivers, which are used to drive the communication interfaces connected to the test host. Of course, a test environment can also be built in the artificial intelligence chip. For example, based on the Linux operating system, create a test environment; and install third-party libraries. For example, for GPU-Tensor core chip testing, you need to install TensorRT related libraries, which will not be described in detail.

[0075] The following is an introduction to the artificial intelligence chip testing method of this application from the test host side.

[0076] like Figure 1 , showing a flow chart of the artificial intelligence chip testing method provided by the present application. This embodiment is applied to a test host, and there is a communication connection between the test host and the artificial intelligence chip to be tested. The details are as described above and will not be repeated here.

[0077] This embodiment may further include the following steps S101-S105:

[0078] S101, sending a first test instruction to the artificial intelligence chip.

[0079] Among them, the first test instruction is used to instruct the artificial intelligence chip to perform target task reasoning based on the target artificial intelligence model. The target artificial intelligence model belongs to at least one artificial intelligence model deployed in the artificial intelligence chip. In this application, the first test instruction is an instruction for instructing to test reasoning computing power.

[0080] Among them, target task reasoning refers to the reasoning performed by the artificial intelligence model to execute the target task. For example, target task reasoning can include but is not limited to image classification tasks, object detection tasks, image detection tasks, semantic segmentation tasks, point cloud detection tasks, speech recognition tasks, and reasoning required for autonomous driving tasks.

[0081] For example, the first test instruction indicates the name of the target artificial intelligence model and the name of the target task to be performed with the help of the target artificial intelligence model, so that the target artificial intelligence model called by the artificial intelligence chip can perform reasoning corresponding to the target task.

[0082] S102, sending at least one first test data to the artificial intelligence chip.

[0083] Among them, the first test data is the test data that needs to be processed by the target artificial intelligence model to perform target task reasoning.

[0084] In the present application, the first test data may be test data in a test data set that matches the target artificial intelligence model and the target task reasoning. The test data set may include multiple test data. For ease of distinction, when the first test instruction is sent to the artificial intelligence chip, the test data sent to the artificial intelligence chip is referred to as the first test data.

[0085] The test data sets required for different artificial intelligence models and reasoning for different target tasks may also be different. For any artificial intelligence model and target task, the test data set may be a common test data set in the current field of artificial intelligence technology.

[0086] For example, for image classification tasks, the artificial intelligence model that needs to be tested can be a model such as Resnet18 or Resnet50, and the test dataset can be the Imagenet dataset. The Imagenet dataset is a large visual database used for visual object recognition software research, which can contain most common objects and scenes in life.

[0087] For target detection tasks, the artificial intelligence model that needs to be tested can be a model such as YoloP or InternImage, and the test dataset can be the BDD100K dataset, which can include a large number of driving video images.

[0088] For semantic segmentation tasks, the AI ​​models that need to be tested can be STDC-Seg or SegFormer models, and the test dataset can be the Cityscapes dataset. Cityscapes is a large semantic segmentation dataset derived from stereo video sequences shot by dual cameras.

[0089] For point cloud detection tasks, the AI ​​models that need to be tested can be PointPillars, Sencond or PartA2 models, and the test datasets can be KITTI datasets or NuScenes datasets. KITTI dataset is a large dataset for visual measurement research, which contains a series of 3D object detection and recognition data; NuScenes dataset is a large 3D scene understanding dataset used for autonomous driving and robotics research.

[0090] For speech recognition tasks, the artificial intelligence model that needs to be tested can be a model such as RNNT or Conformer, and the test data set can be the Aishell-1 data set, which can include a large amount of data required for training the speech recognition model.

[0091] For autonomous driving tasks, the artificial intelligence model that needs to be tested may be a model such as Transfuser, and the test data set may be a CARLA data set, which may include a large amount of data reflecting driver behavior.

[0092] It is understandable that in order to test the computing power of the artificial intelligence chip more reasonably and effectively, the test host generally sends multiple first test data to the artificial intelligence chip in batches.

[0093] It should be noted that the order of the above steps S101 and S102 is not limited to Figure 1 As shown, in actual application, steps S101 and S102 may also be performed synchronously.

[0094] S103: For each first test data, record the sending time of the first test data as the inference start time of the first test data.

[0095] S104, for each first test data, in response to the inference end signal of the first test data returned by the artificial intelligence chip, determine the sending time of the inference end signal as the inference end time of the first test data.

[0096] Among them, the inference completion signal of the first test data is generated when the artificial intelligence chip uses the target artificial intelligence model to complete the target task inference of the first test data.

[0097] For example, for each first test data, when the artificial intelligence chip uses the target artificial intelligence model to complete the target task reasoning for the first test data, the artificial intelligence chip sends an inference end signal to the test host, and the inference end signal carries the number of the first test data or other data identifier.

[0098] In the present application, there are also many possibilities for the specific implementation of determining the time when the inference end signal is issued. For example, a timestamp can be added to the inference end signal, and the timestamp is the timestamp corresponding to the time when the processing of the first test data is completed, and the timestamp can be used as the timestamp corresponding to the time when the inference end signal is issued. For another example, under the premise that there is a communication connection such as GPIO, CAN, SPI or UART serial communication between the artificial intelligence chip and the test host, the artificial intelligence chip can generate and issue the inference end signal by triggering the GPIO rising edge or falling edge, triggering the CAN signal, or triggering the start signal of the data signal line (SDA) on the I2C, etc., so that the test host can detect the issuance time of the inference end signal.

[0099] S105: Determine the effective computing power information of the artificial intelligence chip based on the inference start time and the inference end time of each first test data.

[0100] The effective computing power information may be a computing power grade, a computing power level, or an indicator value of a computing power indicator, without any restriction.

[0101] In the present application, there are many possibilities for the specific implementation of determining the effective computing power information, and there is no limitation on this.

[0102] For example, in a possible implementation, for each first test data, the inference duration of the first test data is determined based on the inference start time and the inference end time of the first test data. On this basis, based on the inference duration of each first test data, the benchmark inference duration required for the artificial intelligence chip to complete the target task inference is determined, and based on the benchmark inference duration and the multiplication and addition operations of the target artificial intelligence model, the effective computing power index value of the artificial intelligence chip is determined.

[0103] The inference duration of the first test data is the time difference between the inference end time of the first test data and the inference start time of the first test data.

[0104] Among them, the multiplication and accumulation operations (MACs) refers to the number of multiplication and accumulation operations, which is an indicator to measure the computational complexity of an artificial intelligence model. In this application, the multiplication and accumulation operations of different artificial intelligence models can be pre-configured.

[0105] Among them, the benchmark inference time can represent the time required for the artificial intelligence chip to complete task inference of a test data using the target artificial intelligence model.

[0106] For example, the benchmark inference duration may be an average of the inference durations of each first test data.

[0107] For another example, the inference time of each first test data can be sorted from small to large, and the inference time at the Nth percentile can be determined as the benchmark inference time required for the artificial intelligence chip to complete the inference of the target task. Wherein, N is an integer not less than 90 and less than 100, such as N can be 90, 95 or 99. Wherein, the inference time at the Nth percentile refers to the value at the N% position in the sorting.

[0108] For example, in order to reflect the time required for an artificial intelligence chip to process most of the test data using an artificial intelligence model, N can be set to 90, and the inference time of each first test data can be sorted from small to large, and the inference time at the 90th position can be determined as the benchmark inference time.

[0109] Of course, there may be other ways to determine the benchmark inference time, and there is no limitation to this.

[0110] Among them, there may be many possibilities for the specific implementation of determining the computing power index value based on the benchmark inference time and the multiplication and addition operations, and this application does not impose any restrictions on this.

[0111] In one possible implementation, the reciprocal of the benchmark inference time can be determined as the equivalent frame rate; based on the multiplication and addition operations of the target artificial intelligence model, the calculation amount of the target artificial intelligence model is determined, for example, the calculation amount can be equal to twice the multiplication and addition operations of the artificial intelligence model. On this basis, the effective computing power index value of the artificial intelligence chip can be determined based on the product of the calculation amount of the target artificial intelligence model and the equivalent frame rate.

[0112] For example, taking TOPS (Tera Operations Per Second) as a computing power indicator, the effective computing power indicator value is the number of TOPS. TOPS is an indicator for evaluating the computing power of a processor. 1TOPS means that the processor can perform one trillion operations per second. On this basis, the effective computing power indicator value can be the product of the computing amount of the target artificial intelligence model and the equivalent frame rate, divided by 1000.

[0113] For example:

[0114] Assume that the artificial intelligence model is a ResNet50 model. During the artificial intelligence chip test, the target task reasoning performed by the ResNet50 model is image classification. At the same time, assume that N is 90, and the target total number of test data is 10,000, that is, the artificial intelligence chip performs reasoning on 10,000 first test data, and obtains the reasoning time of each first test data in turn.

[0115] Based on this, the inference time of the first test data is the time required for the artificial intelligence chip to run the ResNet50 model to complete the image classification of the first test data.

[0116] After obtaining the inference time corresponding to each of the 10,000 first test data, the inference time of the 10,000 first test data can be sorted from small to large. For example, the inference time is sorted as follows: 5.0ms, 5.01ms, 5.1ms, 5.3ms...6.75ms, 6.8ms. Assuming that the inference time at the 90th position is 6.2ms, the benchmark inference time required for the artificial intelligence chip to complete the image classification of a test data is 6.2ms.

[0117] On this basis, first, convert the benchmark inference duration into an equivalent frame rate: 1 / 0.0062=161.3Hz.

[0118] Then, determine the computational complexity of the ResNet50 model, which is the MACs of the ResNet50 model × 2.

[0119] If the MACs of the ResNet50 model is 8.3G, then the effective computing power index value of the AI ​​chip running the ResNet50 model for image classification = 8.3×2×161.3 / 1000=2.68TOPs, where the product of the equivalent frame rate and the amount of computation divided by 1000 is the unit from G (that is, 10 9 ) switches to T (i.e. 10 12 ) conversion.

[0120] Of course, in this application, there may be other possibilities for the computing power indicators used to evaluate the computing power of artificial intelligence chips, which will not be elaborated here.

[0121] From the above content, it can be seen that in this application, the test host can instruct the artificial intelligence chip to call the target artificial intelligence model to perform the specified target task reasoning by sending a first test instruction to the artificial intelligence chip. On this basis, the test host will send the first test data to the artificial intelligence chip, and record the sending time of the first test data as the reasoning start time of the first test data. Moreover, after the artificial intelligence chip completes the target task reasoning of the first test data and returns the reasoning end signal, the test host will determine the sending time of the reasoning end signal as the reasoning end time of the first test data. Since the reasoning start time and reasoning end time of the first test data reflect the actual time taken by the artificial intelligence chip to actually run the artificial intelligence model to process the first test data, therefore, combined with the reasoning start time and reasoning end time of each first test data, the actual effective computing power of the artificial intelligence chip can be more accurately determined.

[0122] In the present application, there may be many possibilities for the specific implementation of the test host sending the at least one first test data to the artificial intelligence chip. For example, at least one first test data may be sent to the artificial intelligence chip in sequence.

[0123] In a possible implementation, the present application may also combine the hardware capabilities of the artificial intelligence chip to be tested to send the first test data to the artificial intelligence model in batches. Figure 2 , shows a schematic diagram of an implementation process of sending the first test data from the test host to the artificial intelligence chip in the present application, the process is applied to the test host, and the process may include:

[0124] S201, determine the data sending frame rate configured for the artificial intelligence chip, the set number of test data to be sent each time, the target number of test data to be selected, and the target total number of test data required.

[0125] In the present application, the data transmission frame rate suitable for the artificial intelligence chip to be tested, the number of test data to be sent each time (i.e., the set number), the target number of test data to be selected from the test data set, and the target total number of test data required to test the artificial intelligence chip can be configured in the test host in advance according to the hardware performance of the artificial intelligence chip such as memory and bandwidth.

[0126] For example, the data transmission frame rate may be 1, 10, 30, 60 or 120 Hz.

[0127] The set number of data sent each time (batch size) is the number of test data sent in batches each time, such as batch size can be 1, 2, 4, 8, 16, 32, 64, 128, 216, 512 or 1024. For example, batch size = 8 means that 8 test data can be sent simultaneously each time.

[0128] The target number of test data selected can be 64, 128, or 216, etc.

[0129] The target total number of test data required to test the artificial intelligence chip can be 1,000, 10,000, or 100,000, etc.

[0130] S202, determining a target number of first test data from the test data set.

[0131] For example, a target number of first test data may be randomly selected from the test data set. In the present application, the target number is less than the target total number, and is not less than the set number of test data sent each time.

[0132] The specific value of the target number can be set according to actual needs. For example, the target number can be 64, and then 64 first test data need to be selected from the test data set.

[0133] S203, determining a set number of first test data to be sent according to the sequence of the target number of first test data.

[0134] It is understandable that the target number is less than the total target number. Therefore, in order to send no less than the total target number of first test data to the artificial intelligence chip, the present application will repeatedly execute step S203. Based on this, each time step S203 is executed, the test host will select the set number of first test data to be sent from the last determined set number of first test data to be sent in the order of the target number of first test data.

[0135] For example, assuming that the target number is 64 and the set number is 8, when step S203 is executed for the first time, the 1st to 8th first test data will be determined as the first test data to be sent; and when step S203 is executed for the second time, the 9th to 16th first test data will be determined as the first test data to be sent, and so on. On this basis, when step S203 is executed for the eighth time, the 57th to 64th first test data will be determined as the first test data to be sent; when step S203 is executed for the ninth time, the 8 first test data to be sent will be selected from the first first test data in the target number of first test data, that is, the 1st to 8th first test data will be determined as the first test data to be sent, and so on, which will not be repeated.

[0136] S204, when it is determined that the current time has reached the data sending time according to the data sending frame rate, a set number of first test data currently determined are sent to the artificial intelligence chip.

[0137] S205: If the cumulative total number of first test data sent to the artificial intelligence chip has not reached the target total number, return to execute the operation of step S203 until the cumulative total number is not less than the target total number.

[0138] It can be understood that each time step S204 is executed, the cumulative total number of first test data sent by the test host to the artificial intelligence chip will be added to the set number. For example, if the set number is 8, then when step S204 is executed once, the cumulative total number is 8, and when step S204 is executed for the second time, the cumulative total number is 8+8=16. Similarly, when step S204 is executed for the third time, the cumulative total number is 24, and so on.

[0139] Among them, if it is determined that the cumulative total number has not reached the target total number, return to execute step S203; if the cumulative total number reaches or exceeds the target total number, there is no need to send the first test data to the artificial intelligence chip.

[0140] For example:

[0141] Assuming that the target number is 64 and the target total number of test data required is 10,000, the test host may randomly select 64 first test data from the test data set.

[0142] In addition, assuming that the set number of test data sent each time is 8, then according to the sending frame rate, the first test data among the 64 first test data is continuously cycled, and 8 first test data are selected each time to be sent to the artificial intelligence chip until the cumulative total number of first test data sent to the artificial intelligence chip is not less than 10000. Based on this, the 64 first test data need to be sent repeatedly 157 times to make the cumulative total number not less than 10000, that is, 64*157 times=10048.

[0143] It can be understood that, compared with taking out the first test data to be sent from the test data set each time, taking out a target number of first test data from the test data set first, and continuously determining the first test data to be sent from the target number of first test data, can avoid frequent access to the test data set, and can also reduce the efficiency of determining the first test data to be sent due to accessing the test data set, and can reduce resource consumption.

[0144] It can be understood that in the present application, the artificial intelligence chip testing method can be implemented by the test host by running the program code. In order to improve the test performance, the artificial intelligence chip testing method can also be implemented based on multiple hardware modules in the test host.

[0145] For example, in a possible implementation, the test host may include a control module, a data sending module, and a timing monitoring module with communication connections.

[0146] The control module is used to control the data transmission module and the timing monitoring module, and determine the effective computing power information of the artificial intelligence chip. The data transmission module is mainly responsible for data transmission, and the timing monitoring module is mainly responsible for monitoring the time-related information required for the artificial intelligence chip to process the first test data.

[0147] For example, the control module can be used to send a first test instruction to the artificial intelligence chip; and determine the effective computing power information of the artificial intelligence chip based on the inference start time and the inference end time of each first test data.

[0148] The data sending module is used to send at least one first test data to the artificial intelligence chip under the control of the control module.

[0149] The timing monitoring module is used to record the sending time of the first test data as the inference start time of the first test data; and, in response to the inference end signal of the first test data returned by the artificial intelligence chip, determine the sending time of the inference end signal as the inference end time of the first test data.

[0150] The specific implementation of each module performing related operations can be found in the introduction of the previous embodiment, which will not be repeated here.

[0151] For ease of understanding, the artificial intelligence chip testing system provided by this application is introduced below. Figure 3 , showing a schematic diagram of the composition architecture of an artificial intelligence chip testing system provided in an embodiment of the present application.

[0152] Depend on Figure 3 It can be seen that the system includes: a test host 31 and an artificial intelligence chip to be tested 32. As mentioned above, at least one artificial intelligence model is deployed in the artificial intelligence chip.

[0153] The test host 31 includes: a control module 311 with communication connection, a data sending module 312 and a timing monitoring module 313. The control module 311, the data sending module 312 and the timing monitoring module 313 are all in communication connection with the artificial intelligence chip 32.

[0154] For example, there may be a UART serial communication connection or an Ethernet communication connection between the control module and the artificial intelligence chip. The data sending module and the artificial intelligence chip may establish a communication connection based on the MIPI, HDMI or GMSL interface, or an Ethernet communication connection. The timing monitoring module and the artificial intelligence chip may use GPIO, CAN, SPI or UART serial communication, or may establish Ethernet communication. Of course, there may be other possibilities for the communication connection between the control module, the data sending module and the timing monitoring module and the artificial intelligence chip, which are not limited.

[0155] In the present application, the control module 311 is used to send a first test instruction to the artificial intelligence chip. As mentioned above, the first test instruction is used to instruct the artificial intelligence chip to perform target task reasoning based on the target artificial intelligence model.

[0156] The data sending module 312 is used to send at least one first test data to the artificial intelligence chip under the control of the control module.

[0157] For example, when the control module sends the first test instruction to the artificial intelligence chip, it can send the first data sending instruction to the data sending module, and the first data sending instruction is used to instruct the data sending module to send the first test data to the artificial intelligence chip. For example, the first data sending instruction can indicate that there is a data sending mode, and the data sending mode can include the number of first test data to be sent and the sending method, etc.; or, the data sending module is configured with different data sending modes, and the data sending module can send the first test data according to the data sending mode indicated in the test instruction.

[0158] Based on the above, the artificial intelligence chip 32 is used to respond to the first test instruction, call the target artificial intelligence model to perform target task reasoning on the first test data; when completing the target task reasoning on the first test data, send an inference end signal to the timing monitoring module.

[0159] Correspondingly, the timing monitoring module 313 is used to detect that the data sending module sends the first test data to the artificial intelligence chip, and record the sending time of the first test data as the inference start time of the first test data; and, in response to the inference end signal of the first test data returned by the artificial intelligence chip, determine the sending time of the inference end signal as the inference end time of the first test data.

[0160] The control module 311 is also used to determine the effective computing power information of the artificial intelligence chip based on the inference start time and the inference end time of each first test data recorded by the timing monitoring module.

[0161] The specific implementation of the related operations performed by each model in the test host can refer to the related introduction of the previous embodiment, which will not be repeated here.

[0162] It is understandable that in practical applications, in addition to testing the computing power of artificial intelligence chips, it may also be necessary to test the reasoning accuracy of artificial intelligence chips. Among them, reasoning accuracy refers to the accuracy or precision of the reasoning results obtained by the artificial intelligence chip running the artificial intelligence model for reasoning.

[0163] In particular, in order to ensure the accuracy of the computing power test, the AI ​​chip can also be tested for reasoning accuracy before the AI ​​chip is tested for computing power. For example, only when the reasoning accuracy of the AI ​​chip meets the requirements will the user trigger the test host to test the computing power of the AI ​​chip. Of course, in actual applications, the order of reasoning accuracy test and computing power test may not be limited.

[0164] Based on this, the test host can also first determine the test mode that needs to be tested on the artificial intelligence chip. The test mode can be divided into two types: inference computing power mode and inference accuracy test. Figure 4 For explanation. Figure 4 Another flow chart of the artificial intelligence chip testing method of the present application is shown, which is applied to a test host. The method of this embodiment may include:

[0165] S401, determining a test mode for the artificial intelligence chip to be tested.

[0166] For example, AI chips can obtain test patterns input by users.

[0167] For example, if the artificial intelligence chip has not yet been tested for inference accuracy, the artificial intelligence chip determines that it needs to perform an inference accuracy test; otherwise, it confirms that it is necessary to perform an inference computing power test on the artificial intelligence model.

[0168] S402: If the test mode is an inference accuracy test, a second test instruction is sent to the artificial intelligence chip.

[0169] Among them, the second test instruction is used to instruct the artificial intelligence chip to call the target artificial intelligence model to perform target task reasoning on the test data set and determine the reasoning accuracy.

[0170] For example, the second test instruction may indicate: the name of the target artificial intelligence model (other model identifiers), the target task to be performed, the name of the test data set, and information that the test mode is an inference accuracy test. The target task is used to characterize the target task reasoning that needs to be performed using the target artificial intelligence model. For the artificial intelligence model to be tested, the test data set and the target task, please refer to the previous introduction, which will not be repeated here.

[0171] In this embodiment, the test data set sent to the artificial intelligence chip includes at least one second test data and the true value of the task result annotated by the second test data. The true value of the task result annotated by the second test data is the task result that should be obtained theoretically by performing the target task reasoning on the second test data.

[0172] For example, if the target task is a speech recognition task, the true value of the task result annotated by the second test data is the actual speech recognition result of the second test data; if the target task is a target detection task, the true value of the task result annotated by the second test data is the target detection result actually corresponding to the second test data. For example, the target detection result may be whether it contains the target to be detected and the location information of the target to be detected.

[0173] It is understandable that since the inference accuracy test does not need to worry about the inference time, the test data sets required for different task reasoning (or different target tasks) can be deployed in the artificial intelligence chip.

[0174] Of course, in actual applications, the test data set may not be deployed in the artificial intelligence chip due to memory limitations of the artificial intelligence chip or other reasons. Based on this, if the test data set is not deployed in the artificial intelligence chip, the test host can also send each second test data in the test data set to the artificial intelligence chip.

[0175] For example, the test host can send at least one second test data in the test data set to the artificial intelligence chip each time according to the data transmission frame rate corresponding to the reasoning precision test mode, until all the second test data in the test data set are sent to the artificial intelligence chip. For example, the transmission frame rate corresponding to the reasoning precision test mode can be 1, 10, 30, 60 or 120HZ, etc., without limitation. The number of second test data that can be sent to the artificial intelligence chip each time can be 1, 2, 4, 8, 16, 32, 64 or 128, etc., without limitation.

[0176] In one possible implementation, under the premise that the test host includes a control module and a data sending module, the control module may send a second data sending instruction to the data sending module at the same time or after sending the second test instruction to the artificial intelligence chip, and the second data sending instruction is used to instruct the data sending module to send the test data set to the artificial intelligence chip.

[0177] For example, the second data sending instruction is used to indicate that the target task and the test mode are reasoning accuracy tests. Accordingly, the artificial intelligence chip can configure the data to be sent under different target tasks under different test modes, and of course, the data sending method can also be configured. Based on this, the data sending module can request the test data set from the control module based on the reasoning accuracy test and the target task, and send the test data set to the artificial intelligence chip.

[0178] Of course, the second data sending instruction can also indicate the test data set to be sent. Accordingly, the data sending module can obtain the test data set from the control module and send it to the artificial intelligence chip.

[0179] S403, obtaining the inference accuracy result returned by the artificial intelligence chip.

[0180] Among them, the reasoning accuracy result is determined by the artificial intelligence chip based on the true value of the task result of each second test data and the reasoning task result, and the reasoning task result is obtained by the artificial intelligence chip performing target task reasoning on the second test data based on the target artificial intelligence model.

[0181] For example, the artificial intelligence chip responds to the second test instruction, calls the target artificial intelligence model to perform target task reasoning on each second test data, and obtains the reasoning task results of each second test data; and determines the reasoning accuracy result based on the true value of the task result of each second test data and the reasoning task result.

[0182] It is understandable that the indicators for evaluating reasoning accuracy will be different depending on the target task reasoning performed by the artificial intelligence chip using the target artificial intelligence model. Accordingly, the indicators used to reflect the reasoning accuracy in the reasoning accuracy results will also be different.

[0183] For example, if the target task is an image classification task, the inference accuracy result refers to the accuracy of image classification. The corresponding evaluation indicators may include TOP1 and TOP5 indicators. The TOP1 indicator is used to characterize the accuracy of whether the predicted category with the highest probability is consistent with the manually labeled category. The TOP5 indicator refers to the accuracy of whether the top five predicted categories with the highest probability are the same as the manually labeled categories.

[0184] If the target task is an image detection task, then the inference accuracy result refers to the accuracy of image detection, and the corresponding evaluation indicators may include the mAP (Mean Average Precision) indicator.

[0185] If the target task is semantic segmentation, the indicator for evaluating reasoning accuracy can be the MIOU (mean Intersection over Union) indicator;

[0186] If the target task is speech recognition, the indicator for evaluating the inference accuracy can be the WER (Word Error Rate) indicator;

[0187] If the target task is autonomous driving, the indicator for evaluating reasoning accuracy can be the L2-Loss (mean error) indicator, etc.

[0188] Of course, there may be other possible target tasks, and there may be other possible corresponding indicators for evaluating reasoning accuracy, which will not be elaborated here.

[0189] In a possible implementation, if the test host includes a control module, the test mode may be determined by the control module, and the operations of steps S401 to S403 may be performed.

[0190] S404: If the test mode is an inference computing power test, a first test instruction is sent to the artificial intelligence chip.

[0191] Among them, the first test instruction is used to instruct the artificial intelligence chip to perform target task reasoning based on the target artificial intelligence model, and the target artificial intelligence model belongs to at least one artificial intelligence model deployed in the artificial intelligence chip.

[0192] In a possible implementation, step S404 may be performed by a control module of the test host.

[0193] S405, sending at least one first test data to the artificial intelligence chip.

[0194] For the specific implementation of this step, please refer to the previous related introduction.

[0195] In a possible implementation, the control module of the test host can send a first data sending instruction to the data sending module, and the first data sending instruction is used to indicate that the target task and the test mode are reasoning computing power tests. On this basis, the data sending module can respond to the test mode being reasoning computing power tests and send at least one first test data corresponding to the target task to the artificial intelligence chip.

[0196] The specific implementation of the data sending module sending the at least one first test data can be referred to in the previous Figure 2 The relevant introduction of the embodiments will not be repeated here.

[0197] It should be noted that the first test data and the second test data may belong to the same test data set, but in order to facilitate the distinction between test data sent in different test modes, the test data sent in the inference computing power test test mode is referred to as the first test data. Of course, the first test data and the second test data may also belong to different test data sets, and there is no restriction on this.

[0198] S406: Record the sending time of the first test data as the inference start time of the first test data.

[0199] In a possible implementation, each time the data sending module of the test host sends at least one first test data to the artificial intelligence chip, it will notify the timing monitoring module of the sending time corresponding to each first test data sent this time, such as notifying the timing monitoring module of the number of each first test data currently sent and the corresponding sending time. Correspondingly, the timing monitoring module will record the sending time of each first test data as the inference start time of the first test data.

[0200] There are many possibilities for the specific implementation of the data sending module notifying the timing monitoring module of the sending time corresponding to the first test data, and there is no restriction on this. For example, in different application scenarios, the timing monitoring module obtains the sending time of the first test data in different ways. For example, the timing monitoring module can determine the sending time by recording the trigger time of the image / video field synchronization signal Vsync, the time when the laser radar is at a certain set angle during each cycle scan, the trigger time of the sensor periodically sending the signal, the time to trigger the rising edge or falling edge of the GPIO, the time to trigger the CAN signal, or the time to trigger the start state of the SDA on the I2C.

[0201] S407, in response to the inference end signal of the first test data returned by the artificial intelligence chip, determining the sending time of the inference end signal as the inference end time of the first test data.

[0202] Among them, the reasoning end signal is generated by the artificial intelligence chip when it uses the target artificial intelligence model to complete the target task reasoning of the first test data.

[0203] In particular, the artificial intelligence chip can send the reasoning end signal to the timing monitoring module, and accordingly, the timing monitoring module will determine the time when the reasoning end signal is sent as the reasoning end time of the first test data. There are also many possibilities for the specific implementation of the timing monitoring module determining the time when the reasoning end signal is sent, which is not limited.

[0204] For example, there is a GPIO or UART serial interface between the artificial intelligence chip and the timing monitoring module. The artificial intelligence chip can trigger event signals such as the rising edge or falling edge of the GPIO, or trigger the CAN signal, or trigger the start state signal of SDA on I2C, which are convenient for hardware status monitoring. Correspondingly, the timing monitoring module receives the corresponding event signal and determines the GPIO edge trigger time or the trigger time of the start state signal of SDA on I2C as the inference end time of the first test data.

[0205] S408: Determine the effective computing power information of the artificial intelligence chip based on the inference start time and the inference end time of each first test data.

[0206] The specific implementation of the above steps S407 and S408 can refer to the relevant introduction of the previous embodiment, which will not be repeated here.

[0207] It can be understood that in any of the above embodiments, when sending the first test instruction to the artificial intelligence chip, that is, when performing inference computing power testing on the artificial intelligence chip, the artificial intelligence chip can also report the current power value of the artificial intelligence chip to the test host so that the test host can analyze the power consumption of the artificial intelligence chip.

[0208] Specifically, before determining the effective computing power information of the artificial intelligence chip, the present application can also obtain the power values ​​reported by the artificial intelligence chip at different time points. For example, the artificial intelligence chip can send the current power value to the test host according to the set power reporting frequency (such as once per second, etc.). Among them, the power value is the power generated by the artificial intelligence chip currently running the target artificial intelligence model.

[0209] Correspondingly, while or after determining the effective computing power information of the artificial intelligence chip, the test host can also determine the average power and peak power of the artificial intelligence chip in the process of running the target artificial intelligence model based on the power values ​​reported by the artificial intelligence chip at different time points.

[0210] In particular, when the test host includes a control module, the artificial intelligence chip can report the power value to the control module. Accordingly, the control module can determine the average power and the peak power based on the power values ​​reported by the artificial intelligence chip at different time points.

[0211] For example, taking the case where an AI chip needs to undergo inference accuracy testing and inference computing power testing, assuming that the target AI model that the AI ​​chip needs to run is ResNet50 and the target task is an image classification task, then the models, data sets, and possible performance results involved in the test process can be shown in the following table:

[0212]

[0213] In the above table, the name of the artificial intelligence chip to be tested is "XX" as an example.

[0214] Corresponding to an artificial intelligence chip testing method provided in this application, this application also provides an artificial intelligence chip testing device.

[0215] like Figure 5 , shows a schematic diagram of a composition structure of an artificial intelligence chip testing device provided by the present application. The device of this embodiment can be applied to a test host. A communication connection is established between the test host and the artificial intelligence chip to be tested. At least one artificial intelligence model is deployed in the artificial intelligence chip. The device includes:

[0216] A first instruction sending unit 501 is used to send a first test instruction to the artificial intelligence chip, wherein the first test instruction is used to instruct the artificial intelligence chip to perform target task reasoning based on a target artificial intelligence model, and the target artificial intelligence model belongs to at least one artificial intelligence model deployed in the artificial intelligence chip;

[0217] A data sending unit 502, configured to send at least one first test data to the artificial intelligence chip;

[0218] A time recording unit 503, configured to record the sending time of the first test data as the inference start time of the first test data;

[0219] a time determination unit 504, configured to determine, in response to an inference completion signal of the first test data returned by the artificial intelligence chip, a time at which the inference completion signal is sent as an inference completion time of the first test data, wherein the inference completion signal is generated when the artificial intelligence chip completes the target task inference of the first test data using the target artificial intelligence model;

[0220] The computing power determination unit 505 is used to determine the effective computing power information of the artificial intelligence chip based on the inference start time and the inference end time of each first test data.

[0221] In a possible implementation, the computing power determination unit includes:

[0222] A first determining subunit is used to determine, for each first test data, an inference duration of the first test data based on an inference start time and an inference end time of the first test data;

[0223] A second determination subunit is used to determine a benchmark inference time required for the artificial intelligence chip to complete the target task inference based on the inference time of each first test data;

[0224] The computing power determination subunit is used to determine the effective computing power index value of the artificial intelligence chip based on the benchmark inference time and the multiplication and addition operations of the target artificial intelligence model.

[0225] In another possible implementation, the second determining subunit is specifically configured to determine the inference time at the Nth percentile as the benchmark inference time required for the artificial intelligence chip to complete the target task inference according to the order of the inference time of each first test data from small to large, where N is an integer not less than 90 and less than 100;

[0226] The computing power determines the sub-units, including:

[0227] A frame rate conversion subunit, configured to determine the reciprocal of the benchmark inference duration as an equivalent frame rate;

[0228] A computation amount determination subunit, used to determine the computation amount of the target artificial intelligence model based on the multiplication and addition operations of the target artificial intelligence model;

[0229] The computing power value determination subunit is used to determine the effective computing power index value of the artificial intelligence chip based on the product of the computing amount of the target artificial intelligence model and the equivalent frame rate.

[0230] In yet another possible implementation, the first instruction sending unit includes:

[0231] A parameter determination subunit, used to determine the data transmission frame rate configured for the artificial intelligence chip, the set number of test data to be sent each time, the target number of test data to be selected, and the target total number of test data required;

[0232] A data selection subunit, used for determining a target number of first test data from the test data set;

[0233] A data determination subunit, configured to determine the set number of first test data to be sent currently according to the sequence of the target number of first test data;

[0234] A data sending subunit is used to send the set number of first test data currently determined to the artificial intelligence chip when it is determined that the current time reaches the data sending time according to the data sending frame rate;

[0235] The data control subunit is used to return to execute the operation of the data determination subunit if the cumulative total number of first test data sent to the artificial intelligence chip has not reached the target total number until the cumulative total number is not less than the target total number.

[0236] In yet another possible implementation, the device further includes:

[0237] A mode determination unit, configured to determine a test mode to be tested for the artificial intelligence chip before the first instruction sending unit sends a first test instruction to the artificial intelligence chip;

[0238] A second instruction sending unit is used to send a second test instruction to the artificial intelligence chip if the test mode is an inference accuracy test, wherein the second test instruction is used to instruct the artificial intelligence chip to call a target artificial intelligence model to perform target task inference on a test data set and determine the inference accuracy, wherein the test data set includes at least one second test data and a true value of a task result annotated by the second test data;

[0239] an accuracy obtaining unit, configured to obtain an inference accuracy result returned by the artificial intelligence chip, wherein the inference accuracy result is determined by the artificial intelligence chip based on the task result true value of each second test data and the inference task result, and the inference task result is obtained by the artificial intelligence chip performing the target task inference on the second test data based on the target artificial intelligence model;

[0240] The first instruction sending unit is specifically used to send a first test instruction to the artificial intelligence chip if the test mode is an inference computing power test.

[0241] In another possible implementation, the device also includes: a data set sending unit, configured to send the test data set to the artificial intelligence chip before the accuracy acquisition unit obtains the inference accuracy result returned by the artificial intelligence chip, if the test data set is not deployed in the artificial intelligence chip.

[0242] In another possible implementation, the device further includes: a power consumption obtaining unit, configured to obtain power values ​​reported by the artificial intelligence chip at different time points before the computing power determining unit determines the effective computing power information of the artificial intelligence chip;

[0243] The power consumption statistics unit is used to determine the average power and peak power of the artificial intelligence chip in the process of running the target artificial intelligence model based on the power values ​​reported by the artificial intelligence chip at different time points, while or after the computing power determination unit determines the effective computing power information of the artificial intelligence chip.

[0244] A computer program product is also provided in an embodiment of the present application, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any one of the artificial intelligence chip testing methods provided in the embodiments of the present application.

[0245] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When one or more computer programs are executed by an electronic device, the electronic device can implement any one of the artificial intelligence chip testing methods provided in the embodiment of the present application.

[0246] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0247] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0248] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0249] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A method for testing an artificial intelligence chip, characterized in that: Applied to a test host, a communication connection is established between the test host and an artificial intelligence chip to be tested, and at least one artificial intelligence model is deployed in the artificial intelligence chip. The method includes: Sending a first test instruction to the artificial intelligence chip, wherein the first test instruction is used to instruct the artificial intelligence chip to perform target task reasoning based on a target artificial intelligence model, where the target artificial intelligence model belongs to at least one artificial intelligence model deployed in the artificial intelligence chip; Sending at least one first test data to the artificial intelligence chip; Recording the sending time of the first test data as the inference start time of the first test data; In response to an inference end signal of the first test data returned by the artificial intelligence chip, determining the issuance time of the inference end signal as the inference end time of the first test data, wherein the inference end signal is generated when the artificial intelligence chip completes the target task inference of the first test data using the target artificial intelligence model; Based on the inference start time and the inference end time of each first test data, the effective computing power information of the artificial intelligence chip is determined.

2. The artificial intelligence chip testing method according to claim 1, characterized in that: The determining of the effective computing power information of the artificial intelligence chip based on the inference start time and the inference end time of each first test data includes: For each first test data, determining the inference duration of the first test data based on the inference start time and the inference end time of the first test data; Determine, based on the inference duration of each first test data, a benchmark inference duration required for the artificial intelligence chip to complete the inference of the target task; Based on the benchmark inference time and the multiplication and addition operations of the target artificial intelligence model, the effective computing power index value of the artificial intelligence chip is determined.

3. The artificial intelligence chip testing method according to claim 2, characterized in that: The determining, based on the inference duration of each first test data, a benchmark inference duration required for the artificial intelligence chip to complete the target task inference comprises: According to the order of the inference time of each first test data from small to large, the inference time at the Nth percentile is determined as the benchmark inference time required for the artificial intelligence chip to complete the target task inference, where N is an integer not less than 90 and less than 100; The determining of the effective computing power index value of the artificial intelligence chip based on the benchmark inference time and the multiplication and addition operation amount of the target artificial intelligence model includes: Determine the reciprocal of the benchmark inference duration as the equivalent frame rate; Determining the computational amount of the target artificial intelligence model based on the multiplication and addition computational amount of the target artificial intelligence model; Based on the product of the computational complexity of the target artificial intelligence model and the equivalent frame rate, the effective computing power index value of the artificial intelligence chip is determined.

4. The artificial intelligence chip testing method according to claim 1, characterized in that: The sending at least one first test data to the artificial intelligence chip comprises: Determine the data transmission frame rate configured for the artificial intelligence chip, the set amount of test data to be sent each time, the target number of test data to be selected, and the total target number of test data required; Determine the target number of first test data from the test data set; Determining the set number of first test data to be sent currently according to the sequence of the target number of first test data; When it is determined that the current moment reaches the data transmission moment according to the data transmission frame rate, sending the currently determined set number of first test data to the artificial intelligence chip; If the cumulative total number of first test data sent to the artificial intelligence chip has not reached the target total number, return to execute the operation of determining the set number of first test data currently to be sent until the cumulative total number is not less than the target total number.

5. The artificial intelligence chip testing method according to claim 1, characterized in that: Before sending the first test instruction to the artificial intelligence chip, the method further includes: Determining a test mode for the artificial intelligence chip to be tested; The sending a first test instruction to the artificial intelligence chip includes: If the test mode is an inference computing power test, sending a first test instruction to the artificial intelligence chip; The artificial intelligence chip testing method further includes: If the test mode is an inference accuracy test, a second test instruction is sent to the artificial intelligence chip, wherein the second test instruction is used to instruct the artificial intelligence chip to call the target artificial intelligence model to perform target task inference on the test data set and determine the inference accuracy, wherein the test data set includes at least one second test data and the true value of the task result annotated by the second test data; Obtain an inference accuracy result returned by the artificial intelligence chip, wherein the inference accuracy result is determined by the artificial intelligence chip based on the task result true value of each second test data and the inference task result, and the inference task result is obtained by the artificial intelligence chip performing the target task inference on the second test data based on the target artificial intelligence model.

6. The artificial intelligence chip testing method according to claim 5, characterized in that: Before obtaining the inference accuracy result returned by the artificial intelligence chip, it also includes: If the test data set is not deployed in the artificial intelligence chip, each second test data in the test data set is sent to the artificial intelligence chip.

7. The artificial intelligence chip testing method according to claim 1, characterized in that: Before determining the effective computing power information of the artificial intelligence chip, it also includes: Obtaining power values ​​reported by the artificial intelligence chip at different time points; While or after determining the effective computing power information of the artificial intelligence chip, the method further includes: Based on the power values ​​reported by the artificial intelligence chip at different time points, the average power and peak power of the artificial intelligence chip during the operation of the target artificial intelligence model are determined.

8. An artificial intelligence chip testing device, characterized in that: Applied to a test host, a communication connection is established between the test host and the artificial intelligence chip to be tested, at least one artificial intelligence model is deployed in the artificial intelligence chip, and the device includes: A first instruction sending unit, configured to send a first test instruction to the artificial intelligence chip, wherein the first test instruction is used to instruct the artificial intelligence chip to perform target task reasoning based on a target artificial intelligence model, wherein the target artificial intelligence model belongs to at least one artificial intelligence model deployed in the artificial intelligence chip; A data sending unit, configured to send at least one first test data to the artificial intelligence chip; a time recording unit, configured to record the sending time of the first test data as the inference start time of the first test data; a time determination unit, configured to determine, in response to an inference completion signal of the first test data returned by the artificial intelligence chip, a time at which the inference completion signal is issued as an inference completion time of the first test data, wherein the inference completion signal is generated when the artificial intelligence chip completes the target task inference of the first test data using the target artificial intelligence model; The computing power determination unit is used to determine the effective computing power information of the artificial intelligence chip based on the inference start time and the inference end time of each first test data.

9. An artificial intelligence chip testing system, characterized in that: include: A test host and an artificial intelligence chip to be tested, wherein at least one artificial intelligence model is deployed in the artificial intelligence chip; The test host comprises: a control module, a data sending module and a timing monitoring module with communication connection; The control module, data transmission module and timing monitoring module are all in communication connection with the artificial intelligence chip; The control module is used to send a first test instruction to the artificial intelligence chip, wherein the first test instruction is used to instruct the artificial intelligence chip to perform target task reasoning based on a target artificial intelligence model, and the target artificial intelligence model belongs to at least one artificial intelligence model deployed in the artificial intelligence chip; The data sending module is used to send at least one first test data to the artificial intelligence chip under the control of the control module; The artificial intelligence chip is used to respond to the first test instruction, call the target artificial intelligence model to perform the target task reasoning on the first test data; when the target task reasoning on the first test data is completed, send a reasoning end signal to the timing monitoring module; The timing monitoring module is configured to detect that the data sending module sends the first test data to the artificial intelligence chip, record the sending time of the first test data as the inference start time of the first test data; and, in response to an inference end signal of the first test data returned by the artificial intelligence chip, determine the sending time of the inference end signal as the inference end time of the first test data; The control module is also used to determine the effective computing power information of the artificial intelligence chip based on the inference start time and the inference end time of each first test data recorded by the timing monitoring module.

10. The artificial intelligence chip testing system according to claim 9, characterized in that: The control module is also used to determine the test mode to be tested of the artificial intelligence chip before sending the first test instruction to the artificial intelligence chip; if the test mode is an inference accuracy test, send a second test instruction to the artificial intelligence chip, the second test instruction is used to instruct the artificial intelligence chip to call the target artificial intelligence model to perform target task inference on the test data set and determine the inference accuracy, the test data set includes at least one second test data, and the task result true value marked by the second test data; obtain the inference accuracy result returned by the artificial intelligence chip; The artificial intelligence chip is further used to, in response to the second test instruction, call the target artificial intelligence model to perform the target task reasoning on the second test data to obtain the reasoning task result of the second test data; Determine the reasoning accuracy result based on the task result true value of each second test data and the reasoning task result; When the control module sends the first test instruction to the artificial intelligence chip, it is specifically used to send the first test instruction to the artificial intelligence chip if the test mode is an inference computing power test.

Images