An artificial intelligence chip testing method, device and system

Abstract

The application discloses an artificial intelligence chip testing method, device and system, and relates to the technical field of artificial intelligence, and comprises the following steps: a test host sends a first test instruction and at least one first test data to an artificial intelligence chip, wherein the first test instruction is used for instructing the artificial intelligence chip to execute target task reasoning based on a target artificial intelligence model; the test host records the sending time of the first test data as the reasoning start time of the first test data, and determines the sending time of a reasoning end signal as the reasoning end time of the first test data in response to the reasoning end signal returned by the artificial intelligence chip; and based on the reasoning start time and the reasoning end time of each first test data, effective computing power information of the artificial intelligence chip is determined. The scheme of the application can accurately determine the effective computing power of the artificial intelligence chip.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to an artificial intelligence chip testing method, device and system. BACKGROUND

[0002] An artificial intelligence (AI) chip, also referred to as an AI accelerator, is a chip specially used for processing AI-related tasks.

[0003] With the continuous development of AI technology, the computing power of AI models is continuously improved. However, the computing power of AI models is limited by the computing and storage capabilities of AI chips running the AI models. In general, in the case of AI chips running AI models, the effective computing power of the AI chips is much lower than the nominal computing power of the AI chips. Based on this, how to accurately determine the effective computing power (i.e., commonly known as the computing power of the artificial intelligence chip) that the artificial intelligence chip can actually achieve when running the artificial intelligence model is a technical problem that needs to be solved by those skilled in the art. SUMMARY

[0004] In view of the above problems, the present application provides an artificial intelligence chip testing method, device and system, which can accurately determine the effective computing power of the artificial intelligence chip.

[0005] In one aspect, the present application provides an artificial intelligence chip testing method applied to a test host, wherein the test host and a to-be-tested artificial intelligence chip have a communication connection established therebetween, the artificial intelligence chip has at least one artificial intelligence model deployed therein, and the method comprises:

[0006] sending a first test instruction to the artificial intelligence chip, wherein the first test instruction is used to instruct the artificial intelligence chip to perform target task reasoning based on a target artificial intelligence model, and the target artificial intelligence model belongs to the at least one artificial intelligence model deployed in the artificial intelligence chip;

[0007] sending at least one first test data to the artificial intelligence chip;

[0008] recording the sending time of the first test data as the reasoning start time of the first test data;

[0009] determining the sending time of the reasoning end signal as the reasoning end time of the first test data in response to the reasoning end signal of the first test data returned by the artificial intelligence chip, wherein the reasoning end signal is generated by the artificial intelligence chip when completing the target task reasoning on the first test data by using the target artificial intelligence model;

[0010] Determine effective computing power information of the artificial intelligence chip based on the inference start time and the inference end time of each first test data.

[0011] In a possible implementation, the determining of the effective computing power information of the artificial intelligence chip based on the inference start time and the inference end time of each first test data comprises:

[0012] For each first test data, determine an inference duration of the first test data based on the inference start time and the inference end time of the first test data;

[0013] Determine a benchmark inference duration required for the artificial intelligence chip to complete the inference of the target task based on the inference duration of each first test data;

[0014] Determine an effective computing power index value of the artificial intelligence chip based on the benchmark inference duration and a multiply-add operation amount of a target artificial intelligence model.

[0015] In yet another possible implementation, the determining of the benchmark inference duration required for the artificial intelligence chip to complete the inference of the target task based on the inference duration of each first test data comprises:

[0016] Determine, according to the inference duration of each first test data in ascending order, an inference duration at an Nth percentile as the benchmark inference duration required for the artificial intelligence chip to complete the inference of the target task, N being an integer not less than 90 and less than 100;

[0017] The determining of the effective computing power index value of the artificial intelligence chip based on the benchmark inference duration and the multiply-add operation amount of the target artificial intelligence model comprises:

[0018] Determine a reciprocal of the benchmark inference duration as an equivalent frame rate;

[0019] Determine a calculation amount of the target artificial intelligence model based on the multiply-add operation amount of the target artificial intelligence model;

[0020] Determine the effective computing power index value of the artificial intelligence chip based on a product of the calculation amount of the target artificial intelligence model and the equivalent frame rate.

[0021] In yet another possible implementation, the sending of the at least one first test data to the artificial intelligence chip comprises:

[0022] Determine a data sending frame rate configured for the artificial intelligence chip, a set number of test data sent each time, a target number of test data to be selected, and a target total number of test data required;

[0023] determine the target number of first test data from the test data set;

[0024] determine the set number of first test data to be sent currently according to the sequence of the target number of first test data;

[0025] when the data sending time point is reached at the current time point according to the data sending frame rate, send the set number of first test data currently determined to the artificial intelligence chip;

[0026] If the cumulative total number of first test data sent to the artificial intelligence chip has not reached the target total number, return to perform the operation of determining the set number of first test data to be sent currently until the cumulative total number is not less than the target total number.

[0027] In another possible implementation, before sending the first test instruction to the artificial intelligence chip, further comprising:

[0028] determine the test mode to be tested by the artificial intelligence chip;

[0029] The sending of the first test instruction to the artificial intelligence chip comprises:

[0030] If the test mode is inference computing power test, send a first test instruction to the artificial intelligence chip;

[0031] The artificial intelligence chip test method further comprises:

[0032] If the test mode is inference accuracy test, send a second test instruction to the artificial intelligence chip, the second test instruction being used to instruct the artificial intelligence chip to call a target artificial intelligence model to perform target task inference on a test data set, and determine inference accuracy, the test data set comprising at least one second test data and a task result true value labeled by the second test data;

[0033] obtain the inference accuracy result returned by the artificial intelligence chip, the inference accuracy result being determined by the artificial intelligence chip based on the task result true value and the inference task result of each second test data, the inference task result being obtained by the artificial intelligence chip based on the target artificial intelligence model performing the target task inference on the second test data.

[0034] In another possible implementation, before obtaining the inference accuracy result returned by the artificial intelligence chip, further comprising:

[0035] If the test data set is not deployed in the artificial intelligence chip, send each second test data in the test data set to the artificial intelligence chip.

[0036] In yet another possible implementation, before determining the effective computing power information of the artificial intelligence chip, the method further includes:

[0037] obtaining power values reported by the artificial intelligence chip at different time points;

[0038] After determining the effective computing power information of the artificial intelligence chip, the method further includes:

[0039] based on the power values reported by the artificial intelligence chip at different time points, determining the average power and the peak power of the artificial intelligence chip in the process of running the target artificial intelligence model.

[0040] In yet another aspect, the present application also provides an artificial intelligence chip testing device, applied to a test host, a communication connection being established between the test host and an artificial intelligence chip to be tested, at least one artificial intelligence model being deployed in the artificial intelligence chip, and the device including:

[0041] a first instruction sending unit, configured to send a first test instruction to the artificial intelligence chip, the first test instruction being used to instruct the artificial intelligence chip to perform target task reasoning based on a target artificial intelligence model, the target artificial intelligence model belonging to the at least one artificial intelligence model deployed in the artificial intelligence chip;

[0042] a data sending unit, configured to send at least one first test data to the artificial intelligence chip;

[0043] a time recording unit, configured to record a sending time of the first test data as a reasoning start time of the first test data;

[0044] a time determining unit, configured to determine, in response to a reasoning end signal of the first test data returned by the artificial intelligence chip, an issuing time of the reasoning end signal as a reasoning end time of the first test data, the reasoning end signal being generated by the artificial intelligence chip when completing the target task reasoning on the first test data by using the target artificial intelligence model;

[0045] a computing power determining unit, configured to determine, based on the reasoning start time and the reasoning end time of each first test data, the effective computing power information of the artificial intelligence chip.

[0046] In yet another aspect, the present application also provides an artificial intelligence chip testing system, including a test host and an artificial intelligence chip to be tested, at least one artificial intelligence model being deployed in the artificial intelligence chip;

[0047] The test host comprises a control module, a data sending module and a timing monitoring module having a communication connection;

[0048] The control module, the data sending module and the timing monitoring module all have a communication connection with the artificial intelligence chip;

[0049] The control module is configured to send a first test instruction to the artificial intelligence chip, the first test instruction being used to instruct the artificial intelligence chip to perform target task reasoning based on a target artificial intelligence model, the target artificial intelligence model belonging to at least one artificial intelligence model deployed in the artificial intelligence chip;

[0050] The data sending module is configured to send at least one first test data to the artificial intelligence chip under the control of the control module;

[0051] The artificial intelligence chip is configured to respond to the first test instruction, call the target artificial intelligence model to perform the target task reasoning on the first test data, and send an end-of-reasoning signal to the timing monitoring module when the target task reasoning on the first test data is completed;

[0052] The timing monitoring module is configured to detect that the data sending module sends the first test data to the artificial intelligence chip, record the sending time of the first test data as the reasoning start time of the first test data, and respond to the end-of-reasoning signal of the first test data returned by the artificial intelligence chip to determine the sending time of the end-of-reasoning signal as the reasoning end time of the first test data;

[0053] The control module is further configured to determine the effective computing power information of the artificial intelligence chip based on the reasoning start time and the reasoning end time of each first test data recorded by the timing monitoring module.

[0054] In a possible implementation, the control module is further configured to determine a test mode to be tested by the artificial intelligence chip before sending the first test instruction to the artificial intelligence chip; if the test mode is a reasoning accuracy test, send a second test instruction to the artificial intelligence chip, the second test instruction being used to instruct the artificial intelligence chip to call the target artificial intelligence model to perform target task reasoning on a test data set and determine reasoning accuracy, the test data set comprising at least one second test data and a task result true value labeled by the second test data, and obtain the reasoning accuracy result returned by the artificial intelligence chip;

[0055] The artificial intelligence chip is further configured to, in response to the second test instruction, invoke the target artificial intelligence model to perform the target task reasoning on the second test data to obtain a reasoning task result of the second test data; and determine a reasoning accuracy result based on the task result true value and the reasoning task result of each second test data.

[0056] The control module is specifically configured to, when sending the first test instruction to the artificial intelligence chip, send the first test instruction to the artificial intelligence chip if the test mode is the reasoning computing power test.

[0057] From the above, the test host can instruct the artificial intelligence chip to invoke the target artificial intelligence model to perform the specified target task reasoning by sending the first test instruction to the artificial intelligence chip. On this basis, the test host sends the first test data to the artificial intelligence chip and records the sending time of the first test data as the reasoning start time of the first test data. Moreover, after the artificial intelligence chip completes the target task reasoning on the first test data and returns the reasoning end signal, the test host determines the sending time of the reasoning end signal as the reasoning end time of the first test data. Since the reasoning start time and the reasoning end time of the first test data reflect the real use time of the artificial intelligence chip in actually running the artificial intelligence model to process the first test data, the real effective computing power of the artificial intelligence chip can be determined more accurately in combination with the reasoning start time and the reasoning end time of each first test data. BRIEF DESCRIPTION OF DRAWINGS

[0058] The above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent as various embodiments of the present disclosure are disclosed in detail with reference to the drawings. Throughout the drawings, the same or similar reference numerals refer to the same or similar elements. It should be understood that the drawings are schematic, and the original and elements are not necessarily drawn according to the scale.

[0059] Figure 1 A flowchart of an artificial intelligence chip testing method provided in the present application;

[0060] Figure 2 A flowchart of the test host sending the first test data to the artificial intelligence chip in the present application;

[0061] Figure 3 A component architecture diagram of an artificial intelligence chip testing system provided in the present application;

[0062] Figure 4 Another flowchart of an artificial intelligence chip testing method provided in the present application;

[0063] Figure 5A schematic diagram of one of the constituent structures of the artificial intelligence chip testing device provided in the present application. DETAILED DESCRIPTION

[0064] The embodiments of the present application are described below with reference to the accompanying drawings. The terms used in the embodiment part of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application. It is known to those skilled in the art that as technology develops and new scenarios appear, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0065] The terms "first", "second", and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, and this is only a way of distinguishing the objects with the same attributes used in the description of the embodiments of the present application. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, so that the processes, methods, systems, products or equipment containing a series of units do not have to be limited to those units, but can include other units not clearly listed or inherent to these processes, methods, products or equipment.

[0066] The present application is suitable for testing the performance of the artificial intelligence chip to be tested by the test host, wherein the performance test at least includes the computing power test of the artificial intelligence chip.

[0067] In the present application, a communication connection is established between the test host and the artificial intelligence chip to be tested.

[0068] Among them, the communication connection between the test host and the artificial intelligence chip can have many possibilities, and the communication connection between the test host and the artificial intelligence chip has one or more, which is not limited.

[0069] For example, the communication connection between the test host and the artificial intelligence chip can include, but is not limited to, serial communication based on one or more of the following: Universal Asynchronous Receiver / Transmitter (UART), Mobile Industry Processor Interface (MIPI), High Definition Multimedia Interface (HDMI), Gigabit Multimedia Serial Links (GMSL), General-purpose input / output (GPIO), Controller Area Network (CAN), and Serial Peripheral Interface (SPI); the communication connection can also be an Ethernet-based communication connection.

[0070] In this application, the artificial intelligence chip to be tested can be any chip capable of running an artificial intelligence (AI) model. For example, the artificial intelligence chip includes various AI processor chips such as Tensor Processing Unit (TPU) chips, Neural Processing Unit (NPU) chips, Graphics Processing Unit (GPU) chips, Brain Processing Unit (BPU), and Deep Learning Processing Unit (DPU).

[0071] The artificial intelligence chip can be a chip mounted in any device. For example, the target device carrying the artificial intelligence chip can be a terminal, a vehicle terminal, or a server, etc., without limitation.

[0072] Among them, at least one artificial intelligence model is deployed in the artificial intelligence chip. The artificial intelligence model deployed in the artificial intelligence chip can be deployed according to actual needs, without limitation. For example, the artificial intelligence model deployed in the artificial intelligence chip includes, but is not limited to, AI algorithm models such as Resnet18, Resnet50, Yolo, SSD, and VGG.

[0073] In one possible implementation, a test program can be deployed in the artificial intelligence chip, and the test program includes at least one artificial intelligence model. Of course, the test program can also be configured with software code for performing different inference tasks based on each artificial intelligence model.

[0074] It can be understood that in actual application, before testing the artificial intelligence chip, the artificial intelligence chip also needs to be powered on, and various interface drivers are installed for driving various communication interfaces connected with the test host. Of course, a test environment can also be built in the artificial intelligence chip. For example, a test environment is created based on a Linux operating system, and third-party libraries are installed. For example, for GPU-Tensor core chip testing, TensorRT related libraries need to be installed, and details are not repeated.

[0075] The artificial intelligence chip testing method of the present application will be introduced from the test host side first.

[0076] As Figure 1 , a flowchart of the artificial intelligence chip testing method provided by the present application is shown, and the present embodiment is applied to a test host which has a communication connection with the artificial intelligence chip to be tested, as described above, and details are not repeated.

[0077] The present embodiment can further include the following steps S101-S105:

[0078] S101, a first test instruction is sent to the artificial intelligence chip.

[0079] The first test instruction is used to instruct the artificial intelligence chip to perform target task reasoning based on a target artificial intelligence model. The target artificial intelligence model belongs to at least one artificial intelligence model deployed in the artificial intelligence chip. In the present application, the first test instruction is an instruction for indicating test reasoning power.

[0080] The target task reasoning refers to the reasoning performed by the artificial intelligence model for the target task. For example, the target task reasoning can include but is not limited to the reasoning required for image classification tasks, target detection tasks, image detection tasks, semantic segmentation tasks, point cloud detection tasks, speech recognition tasks, and automatic driving tasks.

[0081] For example, the first test instruction indicates the name of the target artificial intelligence model and the name of the target task performed by the target artificial intelligence model, so that the target artificial intelligence model called by the artificial intelligence chip performs reasoning corresponding to the target task.

[0082] S102, at least one first test data is sent to the artificial intelligence chip.

[0083] The first test data is test data required for the target artificial intelligence model to perform target task reasoning.

[0084] In the present application, the first test data can be test data in a test data set matched with the target artificial intelligence model and the target task reasoning. The test data set can include a plurality of test data. For the sake of distinction, the test data sent to the artificial intelligence chip in the case of sending the first test instruction to the artificial intelligence chip is referred to as the first test data.

[0085] Different artificial intelligence models and test data sets required for reasoning of different target tasks will also be different. For any artificial intelligence model and target task, the test data set can adopt the test data set commonly used in the current artificial intelligence technology field.

[0086] For example, for the image classification task, the artificial intelligence model required for testing can be Resnet18 or Resnet50 model, and the test data set can be Imagenet data set. Imagenet data set is a large visual database for visual object recognition software research, which can include most common objects and scenes in life.

[0087] For the target detection task, the artificial intelligence model required for testing can be YoloP or InternImage model, and the test data set can be BDD100K data set. BDD100K data set can include a large number of driving video images.

[0088] For the semantic segmentation task, the artificial intelligence model required for testing can be STDC-Seg or SegFormer model, and the test data set can be Cityscapes data set. Cityscapes is a large semantic segmentation data set derived from stereo video sequences captured by dual cameras.

[0089] For the point cloud detection task, the artificial intelligence model required for testing can be PointPillars, Sencond or PartA2 model, and the test data set can be KITTI data set or NuScenes data set. KITTI data set is a large data set for visual measurement research, which contains a series of three-dimensional object detection and recognition data; NuScenes data set is a large three-dimensional scene understanding data set for autonomous driving and robot technology research.

[0090] For the speech recognition task, the artificial intelligence model required for testing can be RNNT or Conformer model, and the test data set can be Aishell-1 data set. Aishell-1 data set can include a large amount of data required for training a speech recognition model.

[0091] For the automatic driving task, the artificial intelligence model to be tested can be a model such as Transfuser, and the test data set can be a CARLA data set, which can include a large amount of data reflecting the behavior of a driver.

[0092] It can be understood that, in order to more reasonably and effectively test the computing power of the artificial intelligence chip, the test host generally sends a plurality of first test data to the artificial intelligence chip in batches.

[0093] It should be noted that the order of the above steps S101 and S102 is not limited to Figure 1 As shown in the figure, in actual application, the steps S101 and S102 can also be executed synchronously.

[0094] S103, for each first test data, the sending time of the first test data is recorded as the inference start time of the first test data.

[0095] S104, for each first test data, in response to the inference end signal of the first test data returned by the artificial intelligence chip, the sending time of the inference end signal is determined as the inference end time of the first test data.

[0096] The inference end signal of the first test data is generated by the artificial intelligence chip when the artificial intelligence chip completes the inference of the target task on the first test data by using the target artificial intelligence model.

[0097] For example, for each first test data, when the artificial intelligence chip completes the inference of the target task on the first test data by using the target artificial intelligence model, the artificial intelligence chip sends the inference end signal to the test host, and the inference end signal carries the number or other data identifier of the first test data.

[0098] In the present application, the specific implementation of determining the sending time of the inference end signal can also have many possibilities. For example, a timestamp can be added to the inference end signal, which is the timestamp corresponding to the time when the processing of the first test data is completed, and the timestamp can be used as the timestamp corresponding to the sending time of the inference end signal. For another example, on the premise that the artificial intelligence chip and the test host have communication connection such as GPIO, CAN, SPI or UART serial port communication, when the artificial intelligence chip generates and sends the inference end signal, it can trigger the rising or falling edge of GPIO, trigger the CAN signal, or trigger the start signal of the I2C data signal line (SDA), so that the test host detects the sending time of the inference end signal.

[0099] S105, based on the inference start time and the inference end time of each first test data, the effective computing power information of the artificial intelligence chip is determined.

[0100] The effective computing power information can be a computing power level, a computing power level, or an index value of a computing power index, without limitation.

[0101] In the present application, there are many possible specific implementations for determining the effective computing power information, without limitation.

[0102] For example, in a possible implementation, for each first test data, the inference duration of the first test data is determined based on the inference start time and the inference end time of the first test data. Based on the inference durations of the first test data, a baseline inference duration required for the artificial intelligence chip to complete the target task inference is determined, and based on the baseline inference duration and the multiplication and accumulation operations of the target artificial intelligence model, the effective computing power index value of the artificial intelligence chip is determined.

[0103] The inference duration of the first test data is the time difference between the inference end time of the first test data and the inference start time of the first test data.

[0104] The multiplication and accumulation operations (MACs) refer to the number of multiplication and accumulation operations, which is an index for measuring the calculation amount of an artificial intelligence model. In the present application, the multiplication and accumulation operations of different artificial intelligence models can be pre-configured.

[0105] The baseline inference duration can represent the duration required for the artificial intelligence chip to complete a test data task inference using the target artificial intelligence model.

[0106] For example, the baseline inference duration can be the average of the inference durations of the first test data.

[0107] For another example, the inference duration at the Nth percentile can be determined as the baseline inference duration required for the artificial intelligence chip to complete the target task inference, according to the ordering of the inference durations of the first test data from small to large. Wherein N is an integer not less than 90 and less than 100, for example, N can be 90, 95 or 99, etc. Wherein the inference duration at the Nth percentile refers to the value at the N% position in the ordering.

[0108] For example, in order to reflect the duration required for the artificial intelligence chip to process most of the test data using the artificial intelligence model, N can be set to 90, and the inference duration at the 90th position can be determined as the baseline inference duration according to the ordering of the inference durations of the first test data from small to large.

[0109] Of course, there can be other ways to determine the baseline inference duration, without limitation.

[0110] The specific implementation of determining the computing power index value based on the benchmark reasoning time length and the multiply-add operation amount can also have multiple possibilities, which are not limited by the present application.

[0111] In a possible implementation, the reciprocal of the benchmark reasoning time length can be determined as an equivalent frame rate; based on the multiply-add operation amount of the target artificial intelligence model, the calculation amount of the target artificial intelligence model is determined, for example, the calculation amount can be equal to twice the multiply-add operation amount of the artificial intelligence model. On this basis, the effective computing power index value of the artificial intelligence chip can be determined based on the product of the calculation amount of the target artificial intelligence model and the equivalent frame rate.

[0112] For example, taking TOPS (Tera Operations Per Second) as an example of a computing power index, the effective computing power index value is the number of TOPS, and TOPS is an index for evaluating the operation capacity of a processor, 1 TOPS representing that the processor can perform one trillion operations per second. On this basis, the effective computing power index value can be the product of the calculation amount of the target artificial intelligence model and the equivalent frame rate, divided by 1000.

[0113] For example:

[0114] Suppose the artificial intelligence model is a ResNet50 model, and the target task reasoning performed by the ResNet50 model in the artificial intelligence chip test process is image classification. At the same time, suppose N is 90, and the target total quantity of test data is 10000, that is, the artificial intelligence chip performs reasoning on 10000 first test data, and the reasoning time length of each first test data is obtained in turn.

[0115] Based on this, the reasoning time length of the first test data is the time length required for the artificial intelligence chip to run the ResNet50 model to complete image classification on the first test data.

[0116] After obtaining the respective reasoning time lengths of the 10000 first test data, the reasoning time lengths of the 10000 first test data can be sorted in ascending order, for example, the sorting of the reasoning time lengths is: 5.0ms, 5.01ms, 5.1ms, 5.3ms, …, 6.75ms, 6.8ms. Assuming that the reasoning time length at the 90th position is 6.2ms, the benchmark reasoning time length required for the artificial intelligence chip to complete image classification of a test data is 6.2ms.

[0117] On this basis, first, the benchmark reasoning time length is converted into an equivalent frame rate: 1 / 0.0062=161.3Hz.

[0118] Then, the calculation amount of the ResNet50 model is determined, and the calculation amount = MACs of the ResNet50 model x 2.

[0119] If the MACs of the ResNet50 model is 8.3G, then the effective computing power index value of the artificial intelligence chip running the ResNet50 model for image classification = 8.3x2x161.3 / 1000 = 2.68TOPs, wherein the product of the equivalent frame rate and the calculation amount is divided by 1000 is the conversion from G (i.e., 10 9 ) to T (i.e., 10 12 ).

[0120] Of course, in the present application, the computing power index for evaluating the computing power of the artificial intelligence chip can also have other possibilities, which will not be described here.

[0121] From the above, in the present application, the test host can instruct the artificial intelligence chip to call the target artificial intelligence model to execute the specified target task reasoning by sending the first test instruction to the artificial intelligence chip. On this basis, the test host sends the first test data to the artificial intelligence chip and records the sending time of the first test data as the reasoning start time of the first test data. Moreover, after the artificial intelligence chip completes the target task reasoning on the first test data and returns the reasoning end signal, the test host determines the sending time of the reasoning end signal as the reasoning end time of the first test data. Since the reasoning start time and the reasoning end time of the first test data reflect the real use time of the artificial intelligence chip actually running the artificial intelligence model to process the first test data, therefore, in combination with the reasoning start time and the reasoning end time of each first test data, the real effective computing power of the artificial intelligence chip can be determined more accurately.

[0122] In the present application, the specific implementation of the test host sending the at least one first test data to the artificial intelligence chip can have multiple possibilities, for example, the at least one first test data can be sent to the artificial intelligence chip in sequence.

[0123] In one possible implementation, the present application can also send the first test data to the artificial intelligence model in batches in combination with the hardware capability of the artificial intelligence chip to be tested. For example, Figure 2 , a flowchart of one implementation of the test host sending the first test data to the artificial intelligence chip in the present application is shown, which is applied to the test host, and the flowchart can include:

[0124] S201, determining the data sending frame rate configured for the artificial intelligence chip, the set number of test data sent each time, the target number of test data to be selected, and the target total number of test data.

[0125] In the present application, the data sending frame rate suitable for the artificial intelligence chip can be configured in the test host in advance according to the hardware performance such as memory and bandwidth of the artificial intelligence chip to be tested, the number of test data sent each time (i.e. the set number), the target number of test data selected from the test data set, and the target total number of test data required for testing the artificial intelligence chip.

[0126] For example, the data sending frame rate can be 1, 10, 30, 60 or 120 Hz.

[0127] The set number of data sent each time, i.e. the number of test data sent each time in batches, can be 1, 2, 4, 8, 16, 32, 64, 128, 216, 512 or 1024, etc. For example, batch size = 8, which means 8 test data can be sent simultaneously each time.

[0128] The target number of test data selected can be 64, 128 or 216, etc.

[0129] The target total number of test data required for testing the artificial intelligence chip can be 1000, 10000 or 100000, etc.

[0130] S202, determining a target number of first test data from the test data set.

[0131] For example, the target number of first test data can be randomly selected from the test data set. In the present application, the target number is less than the target total number and not less than the set number of test data sent each time.

[0132] The specific value of the target number can be set according to actual needs. For example, the target number can be 64, so 64 first test data need to be selected from the test data set.

[0133] S203, determining the set number of first test data to be sent currently according to the order of the target number of first test data.

[0134] It can be understood that the target number is less than the target total number, so in order to send no less than the target total number of first test data to the artificial intelligence chip, the step S203 will be repeatedly executed multiple times. Based on this, each time the step S203 is executed, the test host will select the set number of first test data to be sent currently after the set number of first test data to be sent determined last time according to the order of the target number of first test data.

[0135] For example, if the target number is 64 and the set number is 8, the first execution of step S203 determines the first to eighth first test data as the first test data to be sent, and the second execution of step S203 determines the ninth to sixteenth first test data as the first test data to be sent, and so on. On this basis, the eighth execution of step S203 determines the fifty-seventh to sixty-fourth first test data as the first test data to be sent, and the ninth execution of step S203 selects the first to eighth first test data from the first test data in the target number as the current first test data to be sent, and so on, which will not be repeated here.

[0136] S204, when the current time reaches the data sending time according to the data sending frame rate, sending the set number of first test data currently determined to the artificial intelligence chip.

[0137] S205, if the cumulative total number of first test data sent to the artificial intelligence chip has not reached the target total number, return to execute step S203 until the cumulative total number is not less than the target total number.

[0138] It can be understood that the cumulative total number of first test data sent by the test host to the artificial intelligence chip will increase by the set number each time step S204 is executed. For example, if the set number is 8, then the cumulative total number is 8 after the first execution of step S204, and the cumulative total number is 8+8=16 after the second execution of step S204, and the cumulative total number is 24 after the third execution of step S204, and so on.

[0139] If it is judged that the cumulative total number has not reached the target total number, step S203 is executed; if the cumulative total number reaches or exceeds the target total number, there is no need to send first test data to the artificial intelligence chip.

[0140] For example:

[0141] Assuming that the target number is 64 and the target total number of test data required is 10000, the test host can randomly select 64 first test data from the test data set.

[0142] In addition, assuming that the set number of test data sent each time is 8, then according to the sending frame rate, the first test data in the 64 first test data is repeatedly looped, and 8 first test data are selected each time to send to the artificial intelligence chip until the cumulative total number of first test data sent to the artificial intelligence chip is not less than 10000. Based on this, the 64 first test data all need to be repeatedly sent 157 times to make the cumulative total number not less than 10000, i.e. 64*157 times = 10048.

[0143] It can be understood that, compared with taking out the first test data to be sent from the test data set each time, taking out the target number of first test data from the test data set first and then determining the first test data to be sent from the target number of first test data can avoid frequent access to the test data set, so as to reduce the resource consumption and the influence on the efficiency of determining the first test data to be sent due to accessing the test data set.

[0144] It can be understood that the artificial intelligence chip testing method in the present application can be realized by running program code by the test host. In order to improve the testing performance, the artificial intelligence chip testing method can also be realized based on a plurality of hardware modules in the test host.

[0145] For example, in a possible implementation, the test host can include a control module, a data sending module and a timing monitoring module connected in communication.

[0146] The control module is configured to control the data sending module and the timing monitoring module, and determine the effective computing power information of the artificial intelligence chip. The data sending module is mainly responsible for data sending, and the timing monitoring module is mainly responsible for monitoring the time related information required by the artificial intelligence chip to process the first test data.

[0147] For example, the control module can be configured to send a first test instruction to the artificial intelligence chip, and determine the effective computing power information of the artificial intelligence chip based on the inference start time and the inference end time of each first test data.

[0148] The data sending module is configured to send at least one first test data to the artificial intelligence chip under the control of the control module.

[0149] The timing monitoring module is configured to record the sending time of the first test data as the inference start time of the first test data, and determine the sending time of the inference end signal of the first test data as the inference end time of the first test data in response to the inference end signal of the first test data returned by the artificial intelligence chip.

[0150] The specific implementation of the operation of each module can be referred to the introduction of the foregoing embodiments, which will not be repeated here.

[0151] For ease of understanding, the artificial intelligence chip testing system provided in this application is described below. For example... Figure 3 The diagram illustrates a component architecture of an artificial intelligence chip testing system provided in this application embodiment.

[0152] Depend on Figure 3 As can be seen, the system includes a test host 31 and an artificial intelligence chip 32 to be tested. As mentioned earlier, at least one artificial intelligence model is deployed in the artificial intelligence chip.

[0153] The test host 31 includes a control module 311, a data transmission module 312, and a timing monitoring module 313, all with communication connections. The control module 311, data transmission module 312, and timing monitoring module 313 are all connected to the artificial intelligence chip 32.

[0154] For example, the control module and the AI ​​chip can have a UART serial communication connection or an Ethernet communication connection. The data transmission module and the AI ​​chip can establish a communication connection based on MIPI, HDMI, or GMSL interfaces, or an Ethernet communication connection. The timing monitoring module and the AI ​​chip can use GPIO, CAN, SPI, or UART serial communication, or an Ethernet communication connection. Of course, there are other possibilities for the communication connections between the control module, data transmission module, timing monitoring module, and AI chip, and there are no restrictions on these possibilities.

[0155] In this application, the control module 311 is used to send a first test instruction to the artificial intelligence chip. As described above, the first test instruction is used to instruct the artificial intelligence chip to perform target task inference based on the target artificial intelligence model.

[0156] The data transmission module 312 is used to send at least one first test data to the artificial intelligence chip under the control of the control module.

[0157] For example, while the control module sends a first test command to the AI ​​chip, it can also send a first data transmission command to the data transmission module. This first data transmission command instructs the data transmission module to send first test data to the AI ​​chip. For instance, the first data transmission command could indicate a data transmission mode, which could include the quantity of first test data to be sent and the transmission method; or, the data transmission module could be configured with different data transmission modes, and the data transmission module could send the first test data according to the data transmission mode indicated in the test command.

[0158] Based on the above, the artificial intelligence chip 32 is used to respond to the first test instruction, call the target artificial intelligence model to perform target task inference on the first test data; when the target task inference on the first test data is completed, it sends an inference end signal to the timing monitoring module.

[0159] Accordingly, the timing monitoring module 313 is used to detect when the data sending module sends the first test data to the artificial intelligence chip, record the sending time of the first test data as the inference start time of the first test data; and, in response to the inference end signal of the first test data returned by the artificial intelligence chip, determine the sending time of the inference end signal as the inference end time of the first test data.

[0160] The control module 311 is also used to determine the effective computing power information of the artificial intelligence chip based on the inference start time and inference end time of each first test data recorded by the timing monitoring module.

[0161] The specific implementation of the relevant operations performed by each model in the test host can be found in the previous embodiments, and will not be repeated here.

[0162] Understandably, in practical applications, in addition to testing the computing power of AI chips, it may also be necessary to test their inference accuracy. Inference accuracy refers to the accuracy or precision of the inference results obtained by the AI ​​chip when running AI models.

[0163] Specifically, to ensure the accuracy of computing power testing, an inference accuracy test can be performed on the AI ​​chip before the computing power test. For example, the user will only trigger the test host to perform a computing power test on the AI ​​chip if its inference accuracy meets the requirements. Of course, in practical applications, the order of inference accuracy testing and computing power testing is not limited.

[0164] Based on this, the test host can also first determine the test mode of the AI ​​chip that needs to be tested. This test mode can be divided into two types: inference computing power mode and inference accuracy test. The following will combine... Figure 4 Please provide an explanation. For example... Figure 4 This illustration shows another flowchart of the artificial intelligence chip testing method of this application, applied to a test host. The method in this embodiment may include:

[0165] S401, determine the test mode of the AI ​​chip to be tested.

[0166] For example, AI chips can obtain test patterns input by users.

[0167] For example, if the inference accuracy test has not been performed on the artificial intelligence chip, the artificial intelligence chip determines that the inference accuracy test needs to be performed; otherwise, it is confirmed that the inference power test needs to be performed on the artificial intelligence model.

[0168] S402, if the test mode is the inference accuracy test, a second test instruction is sent to the artificial intelligence chip.

[0169] The second test instruction is used to instruct the artificial intelligence chip to call the target artificial intelligence model to perform the target task inference on the test data set and determine the inference accuracy.

[0170] For example, the second test instruction can indicate the name (other model identifier) of the target artificial intelligence model, the target task to be performed, the name of the test data set, and the information that the test mode is the inference accuracy test. The target task is used to represent the target task inference required to be performed by the target artificial intelligence model. For the artificial intelligence model to be tested, the test data set and the target task can be referred to the foregoing description and will not be described here.

[0171] In this embodiment, the test data set sent to the artificial intelligence chip includes at least one second test data and the task result true value labeled by the second test data. The task result true value labeled by the second test data is the task result that should be obtained in theory by performing the target task inference on the second test data.

[0172] For example, if the target task is a speech recognition task, the task result true value labeled by the second test data is the actual speech recognition result of the second test data; if the target task is an object detection task, the task result true value labeled by the second test data is the actual object detection result corresponding to the second test data, for example, whether the object to be detected is contained and the position information of the object to be detected.

[0173] It can be understood that, since the inference accuracy test does not need to concern the inference time consumption, the test data set required by different task inferences (or different target tasks) can be deployed in the artificial intelligence chip.

[0174] Of course, in actual application, due to the memory limitation of the artificial intelligence chip or other reasons, the test data set may not be deployed in the artificial intelligence chip. Based on this, if the test data set is not deployed in the artificial intelligence chip, the test host can also send each second test data in the test data set to the artificial intelligence chip.

[0175] For example, the test host can send at least one second test data in the test data set to the artificial intelligence chip at a data sending frame rate corresponding to the inference precision test mode each time until all second test data in the test data set are sent to the artificial intelligence chip. For example, the sending frame rate corresponding to the inference precision test mode can be 1, 10, 30, 60, or 120 HZ, etc., which is not limited. The number of second test data that can be sent to the artificial intelligence chip each time can be 1, 2, 4, 8, 16, 32, 64, or 128, etc., which is not limited.

[0176] In a possible implementation, on the premise that the test host includes a control module and a data sending module, the control module can also send a second data sending instruction to the data sending module at the same time or after sending the second test instruction to the artificial intelligence chip, and the second data sending instruction is used to instruct the data sending module to send the test data set to the artificial intelligence chip.

[0177] For example, the second data sending instruction is used to instruct the target task and the test mode to be the inference precision test, and accordingly, the artificial intelligence chip can configure the data required to be sent by different test modes under different target tasks, and of course, the data sending mode can also be configured. Based on this, the data sending module can request the test data set from the control module based on the inference precision test and the target task, and send the test data set to the artificial intelligence chip.

[0178] Of course, the second data sending instruction can also indicate the test data set to be sent, and accordingly, the data sending module can obtain the test data set from the control module and send it to the artificial intelligence chip.

[0179] S403, obtaining the inference precision result returned by the artificial intelligence chip.

[0180] The inference precision result is determined by the artificial intelligence chip based on the task result true value of each second test data and the inference task result, and the inference task result is obtained by the artificial intelligence chip based on the target artificial intelligence model executing the target task inference on the second test data.

[0181] For example, the artificial intelligence chip responds to the second test instruction, calls the target artificial intelligence model to execute the target task inference on each second test data respectively, obtains the inference task result of each second test data, and determines the inference precision result based on the task result true value of each second test data and the inference task result.

[0182] It can be understood that the target task inference executed by the artificial intelligence chip using the target artificial intelligence model is different, the index for evaluating the inference precision will also be different, and accordingly, the index for reflecting the inference precision in the inference precision result will also be different.

[0183] For example, if the target task is an image classification task, the inference accuracy result refers to the accuracy of image classification, and the corresponding evaluation index can include a TOP1 index and a TOP5 index. The TOP1 index is used to represent the accuracy rate of whether the class with the highest predicted probability is consistent with the manually labeled class. The TOP5 index refers to the accuracy rate of whether the top 5 classes with the highest predicted probability are the same as the manually labeled class.

[0184] If the target task is an image detection task, the inference accuracy result refers to the accuracy of image detection, and the corresponding evaluation index can include an mAP (Mean Average Precision) index.

[0185] If the target task is semantic segmentation, the index for evaluating inference accuracy can be an MIOU (mean Intersection over Union) index.

[0186] If the target task is speech recognition, the index for evaluating inference accuracy can be a WER (Word Error Rate) index.

[0187] If the target task is automatic driving, the index for evaluating inference accuracy can be an L2-Loss (mean error) index, etc.

[0188] Of course, the target task can also have other possibilities, and the index for evaluating inference accuracy can also have other possibilities, which will not be described here.

[0189] In a possible implementation, if the test host includes a control module, the test mode can be determined through the control module, and the operations of steps S401 to S403 can be performed.

[0190] S404, if the test mode is inference computing power test, sending a first test instruction to the artificial intelligence chip.

[0191] The first test instruction is used to instruct the artificial intelligence chip to perform target task inference based on the target artificial intelligence model, and the target artificial intelligence model belongs to at least one artificial intelligence model deployed in the artificial intelligence chip.

[0192] In a possible implementation, this step S404 can be performed through the control module of the test host.

[0193] S405, sending at least one first test data to the artificial intelligence chip.

[0194] The specific implementation of this step can be referred to the previous related introduction.

[0195] In a possible implementation, the control module of the test host can send a first data sending instruction to the data sending module, where the first data sending instruction is used to indicate a target task and a test mode being inference computing power test. On this basis, the data sending module can send at least one first test data corresponding to the target task to the artificial intelligence chip in response to the test mode being inference computing power test.

[0196] The specific implementation of the data sending module sending the at least one first test data can refer to the foregoing Figure 2 The related description of the embodiments will not be repeated here.

[0197] It should be noted that the first test data and the second test data can belong to the same test data set, but in order to distinguish the test data sent in different test modes, the test data sent in the inference computing power test mode is referred to as the first test data. Of course, the first test data and the second test data can also belong to different test data sets, which is not limited.

[0198] S406, record the sending time of the first test data as the inference start time of the first test data.

[0199] In a possible implementation, the data sending module of the test host sends at least one first test data to the artificial intelligence chip each time, and at the same time, informs the timing monitoring module of the sending time corresponding to each first test data sent this time, for example, informs the timing monitoring module of the number of each first test data sent currently and the corresponding sending time. Correspondingly, the timing monitoring module records the sending time of each first test data as the inference start time of the first test data.

[0200] The specific implementation of the data sending module informing the timing monitoring module of the sending time corresponding to the first test data can have many possibilities, which is not limited. For example, in different application scenarios, the timing monitoring module obtains the sending time of the first test data in different ways, for example, the timing monitoring module can determine the sending time by recording the triggering time of the image / video field synchronization signal Vsync, the time of laser radar at a certain set angle each time of circular scanning, the triggering time of the periodic signal sent by the sensor, the time of triggering the rising edge or falling edge of GPIO, the time of triggering the CAN signal, or the time of triggering the start state of SDA on I2C.

[0201] S407, in response to the inference end signal of the first test data returned by the artificial intelligence chip, determine the sending time of the inference end signal as the inference end time of the first test data.

[0202] The inference end signal is generated by the artificial intelligence chip when the artificial intelligence chip completes inference on the first test data using the target artificial intelligence model.

[0203] In particular, the artificial intelligence chip can send the inference end signal to the timing monitoring module, and correspondingly, the timing monitoring module determines the time of sending the inference end signal as the inference end time of the first test data. The specific implementation of the timing monitoring module to determine the time of sending the inference end signal can also have multiple possibilities, which are not limited.

[0204] For example, the artificial intelligence chip and the timing monitoring module have a GPIO or UART serial interface, etc. The artificial intelligence chip can trigger a GPIO rising edge or falling edge, or trigger a CAN signal, or trigger an I2C start state signal on SDA, etc. The timing monitoring module receives the corresponding event signal, and determines the GPIO edge trigger time or the I2C start state signal trigger time as the inference end time of the first test data.

[0205] S408, determining the effective computing power information of the artificial intelligence chip based on the inference start time and the inference end time of each first test data.

[0206] The specific implementation of steps S407 and S408 can be referred to the related description of the foregoing embodiments, which will not be repeated here.

[0207] It can be understood that in any of the above embodiments, when the first test instruction is sent to the artificial intelligence chip, that is, when the inference computing power of the artificial intelligence chip is tested, the artificial intelligence chip can also report the current power value of the artificial intelligence chip to the test host, so that the test host can analyze the power consumption of the artificial intelligence chip.

[0208] Specifically, before determining the effective computing power information of the artificial intelligence chip, the application can also obtain the power values reported by the artificial intelligence chip at different time points. For example, the artificial intelligence chip can send the current power value to the test host according to the set power reporting frequency (such as reporting once per second, etc.). The power value is the power generated by the artificial intelligence chip when running the target artificial intelligence model.

[0209] Correspondingly, at the same time or after determining the effective computing power information of the artificial intelligence chip, the test host can also determine the average power and peak power of the artificial intelligence chip in the process of running the target artificial intelligence model based on the power values reported by the artificial intelligence chip at different time points.

[0210] In particular, in the case that the test host includes a control module, the artificial intelligence chip can report the power value to the control module. Correspondingly, the control module can determine the average power and the peak power based on the power values reported by the artificial intelligence chip at different time points.

[0211] For example, taking the inference accuracy test and inference computing power test of the artificial intelligence chip as an example, assuming that the target artificial intelligence model required to be run by the artificial intelligence chip is ResNet50, and the target task is an image classification task, the model, data set and possible performance results involved in the test process can be as follows:

[0212]

[0213] In the above table, the name of the artificial intelligence chip to be tested is taken as an example.

[0214] Corresponding to the artificial intelligence chip testing method provided in the present application, the present application further provides an artificial intelligence chip testing device.

[0215] As Figure 5 , a schematic diagram of a component structure of an artificial intelligence chip testing device provided by the present application is shown, and the device of the present embodiment can be applied to a test host. The test host and the artificial intelligence chip to be tested are in communication connection, at least one artificial intelligence model is deployed in the artificial intelligence chip, and the device includes:

[0216] The first instruction sending unit 501 is configured to send a first test instruction to the artificial intelligence chip, and the first test instruction is used to instruct the artificial intelligence chip to perform target task inference based on a target artificial intelligence model, and the target artificial intelligence model belongs to at least one artificial intelligence model deployed in the artificial intelligence chip;

[0217] The data sending unit 502 is configured to send at least one first test data to the artificial intelligence chip;

[0218] The time recording unit 503 is configured to record the sending time of the first test data as the inference start time of the first test data;

[0219] The time determining unit 504 is configured to determine the sending time of the inference end signal as the inference end time of the first test data in response to the inference end signal of the first test data returned by the artificial intelligence chip, and the inference end signal is generated by the artificial intelligence chip when the target task inference of the first test data is completed by using the target artificial intelligence model;

[0220] The computing power determination unit 505 is configured to determine effective computing power information of the artificial intelligence chip based on the inference start time and the inference end time of each first test data.

[0221] In a possible implementation, the computing power determination unit comprises:

[0222] The first determination sub-unit is configured to determine, for each first test data, an inference duration of the first test data based on the inference start time and the inference end time of the first test data.

[0223] The second determination sub-unit is configured to determine, based on the inference duration of each first test data, a reference inference duration required by the artificial intelligence chip to complete inference of the target task.

[0224] The computing power determination sub-unit is configured to determine an effective computing power index value of the artificial intelligence chip based on the reference inference duration and a multiply-add operation amount of the target artificial intelligence model.

[0225] In another possible implementation, the second determination sub-unit is specifically configured to determine, in a descending order of the inference duration of each first test data, an inference duration at an Nth percentile as the reference inference duration required by the artificial intelligence chip to complete inference of the target task, where N is an integer not less than 90 and less than 100.

[0226] The computing power determination sub-unit comprises:

[0227] The frame rate conversion sub-unit is configured to determine, as an equivalent frame rate, a reciprocal of the reference inference duration.

[0228] The computing amount determination sub-unit is configured to determine, based on the multiply-add operation amount of the target artificial intelligence model, a computing amount of the target artificial intelligence model.

[0229] The computing power value determination sub-unit is configured to determine, based on a product of the computing amount of the target artificial intelligence model and the equivalent frame rate, the effective computing power index value of the artificial intelligence chip.

[0230] In another possible implementation, the first instruction sending unit comprises:

[0231] The parameter determination sub-unit is configured to determine a data sending frame rate configured for the artificial intelligence chip, a set number of test data sent each time, a target number of test data to be selected, and a target total number of test data required.

[0232] The data selection sub-unit is configured to determine, from a test data set, the target number of first test data.

[0233] determining, in sequence of the target number of first test data, the set number of first test data to be currently sent;

[0234] sending, to the artificial intelligence chip, the set number of first test data currently determined when the data sending time point is reached according to the data sending frame rate;

[0235] if the cumulative total number of first test data sent to the artificial intelligence chip has not reached the target total number, returning to perform the operation of the data determining subunit until the cumulative total number is not less than the target total number.

[0236] In yet another possible implementation, the apparatus further includes:

[0237] a mode determining unit configured to determine a test mode to be tested by the artificial intelligence chip before the first instruction sending unit sends the first test instruction to the artificial intelligence chip;

[0238] a second instruction sending unit configured to send a second test instruction to the artificial intelligence chip if the test mode is the inference accuracy test, the second test instruction being used to instruct the artificial intelligence chip to call the target artificial intelligence model to perform target task inference on a test data set and determine inference accuracy, the test data set including at least one second test data, and the second test data being labeled with a task result true value;

[0239] an accuracy obtaining unit configured to obtain an inference accuracy result returned by the artificial intelligence chip, the inference accuracy result being determined by the artificial intelligence chip based on the task result true value of each second test data and an inference task result, the inference task result being obtained by the artificial intelligence chip based on the target artificial intelligence model performing the target task inference on the second test data;

[0240] The first instruction sending unit is specifically configured to send a first test instruction to the artificial intelligence chip if the test mode is the inference computing power test.

[0241] In yet another possible implementation, the apparatus further includes a data set sending unit configured to send the test data set to the artificial intelligence chip if the artificial intelligence chip does not have the test data set deployed before the accuracy obtaining unit obtains the inference accuracy result returned by the artificial intelligence chip.

[0242] In yet another possible implementation, the apparatus further includes a power consumption obtaining unit, configured to obtain power values reported by the artificial intelligence chip at different time points before the effective computing power information of the artificial intelligence chip is determined by the computing power determining unit.

[0243] a power consumption statistical unit, configured to determine average power and peak power of the artificial intelligence chip in running the target artificial intelligence model based on the power values reported by the artificial intelligence chip at different time points at the same time or after the effective computing power information of the artificial intelligence chip is determined by the computing power determining unit.

[0244] The embodiments of the present application further provide a computer program product, which includes computer readable instructions, and when the computer readable instructions run on an electronic device, the electronic device implements any of the artificial intelligence chip testing methods provided by the embodiments of the present application.

[0245] The embodiments of the present application further provide a computer readable storage medium, which carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement any of the artificial intelligence chip testing methods provided by the embodiments of the present application.

[0246] In addition, it should be noted that the apparatus embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or distributed on multiple network units. According to actual needs, part or all of the modules can be selected to achieve the purpose of the embodiments of the present application. In addition, in the apparatus embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which can be implemented as one or more communication buses or signal lines.

[0247] Those skilled in the art can clearly understand that the application can be implemented by means of software plus necessary universal hardware, and of course can also be implemented by means of dedicated hardware including special integrated circuit, special CPU, special memory, special component, etc. Generally, any function completed by computer program can be easily implemented by corresponding hardware, and the specific hardware structure for implementing the same function can also be various, such as analog circuit, digital circuit or special circuit, etc. However, for the application, software program implementation is a better embodiment. Based on such understanding, the technical solution of the application or the part of the application which makes contribution to the prior art can be embodied in the form of software product, which is stored in readable storage medium, such as computer floppy disk, U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a plurality of instructions for making a computer device (which can be personal computer, training device or network device, etc.) execute the method described in various embodiments of the application.

[0248] In the above embodiments, the implementation can be achieved by software, hardware, firmware or any combination thereof, entirely or partially. When implemented by software, the implementation can be achieved in the form of a computer program product, entirely or partially.

[0249] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the flow or function described in the embodiments of the application is generated entirely or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another, for example, the computer instructions can be transmitted from one website, computer, training device or data center to another website, computer, training device or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that can be stored by a computer or a data storage device such as a training device, a data center, etc. integrated with one or more available media sets. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

Claims

1. An artificial intelligence chip testing method, characterized by, The application is applied to a test host, a communication connection is established between the test host and an artificial intelligence chip to be tested, at least one artificial intelligence model is deployed in the artificial intelligence chip, and the method comprises the following steps: sending a first test instruction to the artificial intelligence chip, the first test instruction being used for instructing the artificial intelligence chip to perform target task reasoning based on a target artificial intelligence model, the target artificial intelligence model belonging to the at least one artificial intelligence model deployed in the artificial intelligence chip; sending at least one first test data to the artificial intelligence chip, comprising the following steps: determining a data sending frame rate configured for the artificial intelligence chip, a set number of test data to be sent each time, a target number of test data to be selected and a target total number of test data required; determining the target number of first test data from a test data set; determining the set number of first test data to be sent currently in the order of the target number of first test data; when the data sending frame rate is determined to reach a data sending time at the current time, sending the set number of first test data currently determined to the artificial intelligence chip; if the cumulative total number of first test data sent to the artificial intelligence chip has not reached the target total number, returning to perform the operation of determining the set number of first test data to be sent currently until the cumulative total number is not less than the target total number; recording the sending time of the first test data as the reasoning start time of the first test data; in response to a reasoning end signal of the first test data returned by the artificial intelligence chip, determining the sending time of the reasoning end signal as the reasoning end time of the first test data, the reasoning end signal being generated by the artificial intelligence chip when the target task reasoning of the first test data is completed by using the target artificial intelligence model; determining effective computing power information of the artificial intelligence chip based on the reasoning start time and the reasoning end time of each first test data, comprising the following steps: for each first test data, determining the reasoning duration of the first test data based on the reasoning start time and the reasoning end time of the first test data; according to the ordering of the reasoning duration of each first test data from small to large, determining the reasoning duration at the Nth percentile as a reference reasoning duration required for the artificial intelligence chip to complete the target task reasoning, N being an integer not less than 90 and less than 100; determining an effective computing power index value of the artificial intelligence chip as the effective computing power information based on the reference reasoning duration and the multiply-add operation amount of the target artificial intelligence model.

2. The artificial intelligence chip test method according to claim 1, wherein the effective computing power index value of the artificial intelligence chip is determined based on the reference reasoning duration and the multiply-add operation amount of the target artificial intelligence model, comprising the following steps: determining the reciprocal of the reference reasoning duration as an equivalent frame rate; determining the calculation amount of the target artificial intelligence model based on the multiply-add operation amount of the target artificial intelligence model; ​ Determine an effective computing power index value of the artificial intelligence chip based on a product of a computing amount of the target artificial intelligence model and the equivalent frame rate. 3.The artificial intelligence chip test method of claim 1, wherein, Before sending the first test instruction to the artificial intelligence chip, further comprising: Determine a test mode to be tested by the artificial intelligence chip; The first test instruction sending to the artificial intelligence chip comprises: If the test mode is inference computing power test, send the first test instruction to the artificial intelligence chip; The artificial intelligence chip test method further comprises: If the test mode is inference accuracy test, send a second test instruction to the artificial intelligence chip, the second test instruction is used to instruct the artificial intelligence chip to call a target artificial intelligence model to perform target task inference on a test data set, and determine inference accuracy, the test data set comprises at least one second test data, and the second test data is labeled with a task result true value; Obtain the inference accuracy result returned by the artificial intelligence chip, the inference accuracy result is determined by the artificial intelligence chip based on the task result true value labeled by each second test data and an inference task result, and the inference task result is obtained by the artificial intelligence chip based on the target artificial intelligence model performing the target task inference on the second test data. 4.The artificial intelligence chip test method of claim 3, wherein, Before obtaining the inference accuracy result returned by the artificial intelligence chip, further comprising: If the test data set is not deployed in the artificial intelligence chip, send each second test data in the test data set to the artificial intelligence chip. 5.The artificial intelligence chip testing method of claim 1, wherein, Before determining the effective computing power information of the artificial intelligence chip, further comprising: Obtain the power value reported by the artificial intelligence chip at different time points; Simultaneously or after determining the effective computing power information of the artificial intelligence chip, further comprising: Based on the power value reported by the artificial intelligence chip at different time points, determine the average power and peak power of the artificial intelligence chip in the process of running the target artificial intelligence model.

6. An artificial intelligence chip testing apparatus, characterized by, Applied to a test host, the test host and the artificial intelligence chip to be tested have a communication connection, at least one artificial intelligence model is deployed in the artificial intelligence chip, and the device comprises: A first instruction sending unit is configured to send a first test instruction to the artificial intelligence chip, the first test instruction is used to instruct the artificial intelligence chip to perform target task inference based on a target artificial intelligence model, and the target artificial intelligence model belongs to at least one artificial intelligence model deployed in the artificial intelligence chip; The data sending unit is configured to send at least one first test data to the artificial intelligence chip, including: determining a data sending frame rate configured for the artificial intelligence chip, a set number of test data sent each time, a target number of test data to be selected, and a target total number of test data required; determining the target number of first test data from a test data set; determining the set number of first test data to be sent in the order of the target number of first test data; when the current time reaches the data sending time according to the data sending frame rate, sending the set number of first test data currently determined to the artificial intelligence chip; if the cumulative total number of first test data sent to the artificial intelligence chip has not reached the target total number, returning to perform the operation of determining the set number of first test data to be sent until the cumulative total number is not less than the target total number; The time recording unit is configured to record the sending time of the first test data as the inference start time of the first test data; The time determination unit is configured to determine the sending time of the inference end signal as the inference end time of the first test data in response to the inference end signal returned by the artificial intelligence chip, the inference end signal being generated by the artificial intelligence chip when completing the target task inference of the first test data using the target artificial intelligence model; The computing power determination unit is configured to determine the effective computing power information of the artificial intelligence chip based on the inference start time and the inference end time of each first test data, including: for each first test data, determining the inference duration of the first test data based on the inference start time and the inference end time of the first test data; determining the inference duration at the Nth percentile as the reference inference duration required by the artificial intelligence chip to complete the target task inference according to the ordering of the inference duration of each first test data from small to large, N being an integer not less than 90 and less than 100; determining the effective computing power index value of the artificial intelligence chip as the effective computing power information based on the reference inference duration and the multiply-add operation amount of the target artificial intelligence model.

7. An artificial intelligence chip testing system, characterized by, The test host and the artificial intelligence chip to be tested, the artificial intelligence chip having at least one artificial intelligence model deployed therein; The test host includes a control module, a data sending module, and a timing monitoring module having communication connections; The control module, the data sending module, and the timing monitoring module all have communication connections with the artificial intelligence chip; The control module is configured to send a first test instruction to the artificial intelligence chip, the first test instruction being used to instruct the artificial intelligence chip to perform target task inference based on a target artificial intelligence model, the target artificial intelligence model belonging to the at least one artificial intelligence model deployed in the artificial intelligence chip; ​ The data sending module is configured to send, under control of the control module, at least one first test data to the artificial intelligence chip, including: determining a data sending frame rate configured for the artificial intelligence chip, a set number of test data sent each time, a target number of test data to be selected, and a target total number of test data required; determining the target number of first test data from a test data set; determining the set number of first test data to be sent in the order of the target number of first test data; when the current time reaches the data sending time according to the data sending frame rate, sending the set number of first test data currently determined to the artificial intelligence chip; if the cumulative total number of first test data sent to the artificial intelligence chip has not reached the target total number, returning to perform the operation of determining the set number of first test data to be sent until the cumulative total number is not less than the target total number; The artificial intelligence chip is configured to respond to the first test instruction, call the target artificial intelligence model to perform the target task reasoning on the first test data; and send a reasoning end signal to the timing monitoring module when the target task reasoning on the first test data is completed. The timing monitoring module is configured to record the sending time of the first test data as the reasoning start time of the first test data when detecting that the data sending module sends the first test data to the artificial intelligence chip; and determine the sending time of the reasoning end signal as the reasoning end time of the first test data in response to the reasoning end signal of the first test data returned by the artificial intelligence chip. The control module is further configured to determine effective computing power information of the artificial intelligence chip based on the reasoning start time and the reasoning end time of each first test data recorded by the timing monitoring module, including: for each first test data, determining the reasoning duration of the first test data based on the reasoning start time and the reasoning end time of the first test data; determining the reasoning duration at the Nth percentile as a reference reasoning duration required by the artificial intelligence chip to complete the target task reasoning according to the ordering of the reasoning duration of each first test data from small to large, N being an integer not less than 90 and less than 100; and determining an effective computing power index value of the artificial intelligence chip based on the reference reasoning duration and the multiply-add operation amount of the target artificial intelligence model as the effective computing power information.

8. The artificial intelligence chip testing system of claim 7, wherein, The control module is further configured to determine a test mode to be tested by the artificial intelligence chip before sending the first test instruction to the artificial intelligence chip; if the test mode is inference accuracy testing, send a second test instruction to the artificial intelligence chip, the second test instruction being configured to instruct the artificial intelligence chip to call a target artificial intelligence model to perform target task inference on a test data set, and determine inference accuracy, the test data set comprising at least one second test data and a task result true value labeled by the second test data; and obtain an inference accuracy result returned by the artificial intelligence chip. The artificial intelligence chip is further configured to, in response to the second test instruction, call the target artificial intelligence model to perform the target task inference on the second test data to obtain an inference task result of the second test data. The inference accuracy result is determined based on the task result true value labeled by each second test data and the inference task result. The control module is configured to, when sending the first test instruction to the artificial intelligence chip, send the first test instruction to the artificial intelligence chip if the test mode is inference computing power testing.

Images