Operator detection method and device and related equipment

Abstract

The invention provides an operator detection method and device and related equipment, and the method comprises the following steps: monitoring the operation progress of a training script on an accelerator card, obtaining first input data and a first output result of a to-be-detected operator under the condition that the to-be-detected operator is operated, then subtly disturbing the first input data to obtain second input data, inputting the second input data into a to-be-detected operator, re-operating the to-be-detected operator to obtain a second output result, and comparing a first output result before disturbance with the second output result after disturbance to obtain an operator detection result of the accelerator card. Therefore, the operator detection of the accelerator card does not depend on the benchmark accelerator card any more, the operator detection method is not limited any more, all possible detection scenes can be covered, and the detection requirement of a user is met.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence (AI), and in particular to an operator detection method, apparatus and related equipment. Background Technology

[0002] Accelerator cards are hardware devices used to accelerate computing tasks and are widely used in fields such as AI and big data analytics. Operators within an accelerator card are basic functional modules that perform specific computational operations, such as matrix multiplication and convolution. Since the accuracy of operators directly affects the accuracy of the accelerator card's computational results, operator testing is particularly important. Typically, when testing the operator of an accelerator card under test, a benchmark accelerator card with verified operator accuracy is selected. Then, the same input data and operators are used to perform calculations on both the benchmark and the accelerator card under test. The results are compared to determine whether the operator of the accelerator card under test meets the expected accuracy standard.

[0003] However, this operator detection method relies on benchmark accelerator cards. If the performance of the accelerator card under test is better than that of the benchmark accelerator card, or if some complex operators are difficult to fully reproduce on the benchmark accelerator card, this method will not be usable. This limits the current operator detection method, making it unable to cover all possible detection scenarios and failing to meet the detection needs of users. Summary of the Invention

[0004] This application provides an operator detection method, apparatus, and related equipment to solve the problem that current operator detection relies on benchmark accelerator cards, making it difficult for operator detection to cover all possible detection scenarios and meet user detection needs.

[0005] Firstly, an operator detection method is provided, which includes the following steps: an accelerator card receives a calculation instruction sent by a processor, runs multiple operators based on the calculation instruction, the calculation instruction being generated by the processor according to the training script of an artificial intelligence (AI) model, when running to the operator to be detected, obtains the first input data and the first output result of the operator to be detected, perturbs the first input data to obtain second input data, inputs the second input data into the first operator, runs the first operator to obtain the second output result, and compares the first output result with the second output result to obtain the operator detection result.

[0006] By implementing the method described in the first aspect, the accelerator card monitors the running progress of the training script. When it reaches the operator to be detected, it obtains the first input data and the first output result of the operator to be detected. Then, it performs a slight perturbation on the first input data to obtain the second input data. The second input data is then input into the operator to be detected, and the operator to be detected is run again to obtain the second output result. By comparing the first output result before the perturbation and the second output result after the perturbation, the operator detection result of the accelerator card is obtained. This makes the operator detection of the accelerator card no longer dependent on the benchmark accelerator card, and the operator detection method is no longer limited. It can cover all possible detection scenarios and meet the user's detection needs.

[0007] In one possible implementation, the accelerator card can monitor the running progress of multiple operators based on an operator list. When running to the operator to be tested, it can obtain the first input data and the first output result of the operator to be tested. The operator list includes multiple floating-point operators and multiple hot-spot operators. The floating-point operators include operators used to process floating-point operations, and the hot-spot operators include operators with historical anomalies and operators expected to have anomalies in the future.

[0008] Optionally, the operators to be detected in the operator list can be user-configured. Specifically, the user can input the operators to be detected through a client, and the client sends a configuration request carrying the operators to be detected to the operator detection system. The operator detection system can then add the operators to be detected from the configuration request to the operator list. Typically, the operators configured by the user are hot operators. Users can add some operators that have previously detected problems, as well as some operators that may cause problems, based on experience. This application does not impose specific limitations on this.

[0009] Optionally, the operators to be detected in the operator list can be determined based on the user's training script. Specifically, for floating-point operators, whether an operator is a floating-point operator can be determined based on its name or type. This step can be implemented using APIs provided by machine learning frameworks. For example, the model summary function of TensorFlow and PyTorch can list all operators and their types in the model. For hotspot operators, operators that have historically encountered problems can be identified as hotspot operators based on the accelerator card's historical training logs and error reports. Alternatively, potential hotspot operators can be identified through performance data analysis during training, such as operator execution time and memory usage.

[0010] The above implementation, by pre-configuring an operator list, allows the accelerator card to monitor the progress of operators in real time during the training script execution. When it reaches an operator to be detected, it directly obtains the operator's input data and output results to complete the detection, eliminating the need to run the operator separately for detection, thus reducing the overhead of operator detection. Furthermore, this method of detecting operators simultaneously during training can also identify operators that encounter problems during execution, improving the stability and reliability of model training.

[0011] In one possible implementation, the accelerator card compares the first output result with the second output result to obtain the operator detection result. First, the accelerator card obtains the comparison threshold for the operator to be detected based on a threshold list. Then, it compares the difference between the first and second output results with the comparison threshold to obtain the operator detection result. The operator detection result includes information about abnormal operators. If the difference is greater than the threshold, the operator to be detected is determined to be an abnormal operator. The threshold list includes the comparison threshold corresponding to the operator. The comparison threshold is obtained by the accelerator card after attenuating the initial threshold through multiple rounds of threshold adjustment before receiving the calculation instructions sent by the processor.

[0012] It should be understood that different operators may yield different results after perturbation. For example, some operators will change significantly after slight perturbation, so the comparison threshold for these operators can be higher. On the other hand, some operators will not change significantly after slight perturbation, so the comparison threshold for these operators can be lower.

[0013] The above implementation method pre-uses a multi-round adjustment method to determine the corresponding comparison threshold for each operator to be detected. This ensures that when each operator to be detected is detected, the accuracy of the operator is determined according to different comparison thresholds, making the operator detection results more accurate.

[0014] In one possible implementation, each round of threshold adjustment in the multi-round threshold adjustment method includes the following steps: the accelerator card runs the operator to be detected to obtain a first output result before perturbation and a second output result after perturbation; the accelerator card determines the difference value between the first output result and the second output result; if the comparison result between the difference value and the current round threshold reaches the stopping condition, the accelerator card determines the current round threshold as the comparison threshold of the operator to be detected, wherein the current round threshold includes the initial threshold or the threshold obtained from the previous round threshold adjustment; if the comparison result between the difference value and the current round threshold does not reach the stopping condition, the accelerator card attenuates the current round threshold to obtain the next round threshold.

[0015] In simple terms, an initial threshold is assigned to the operator to be detected, and then multiple rounds of threshold adjustment are performed. In each round, the first output result before the operator is perturbed and the second output result after the perturbation are calculated, the difference between the two is determined, and the difference is compared with the threshold of the current round. If the comparison result meets the stopping condition, the threshold of the current round is used as the comparison threshold of the operator to be detected. If the comparison result does not meet the stopping condition, the threshold of the current round is decayed to obtain the threshold of the next round, and then the process is repeated until the comparison threshold corresponding to the operator to be detected is obtained.

[0016] In practice, the threshold for the next round can be a decay value calculated using a decay formula, such as half the threshold for the current round. Alternatively, the threshold for the next round can be the difference value from the current round. Or, the threshold for the next round can be obtained based on both the decay value and the difference value, for example, the maximum of the decay value and the difference value can be used as the threshold for the next round.

[0017] It should be understood that although operators perform basic addition, subtraction, multiplication, and division operations, the results of each repeated run of the operator may differ during actual computation. The reasons for these differences are complex and varied. For example, on an accelerator card, multiple computing cores may execute tasks in parallel, and the scheduling order and computational parallelism of these cores may lead to slight differences in the results of each run. Furthermore, operators on accelerator cards typically use floating-point arithmetic, which is essentially an approximation. Due to the precision limitations of floating-point numbers, many addition, subtraction, multiplication, and division results may differ. Additionally, the hardware of the accelerator card may optimize floating-point operations, such as by using different rounding modes to improve performance, which can also cause slight variations in the results of each operation. Of course, there are other possible reasons, which will not be listed here.

[0018] The above implementation method runs the operator to be detected multiple times before the user starts training the model to obtain multiple first output results before perturbation and second output results after perturbation. Then, it determines whether to perform threshold decay based on the error value between the two. Since each run may produce different results, multiple rounds of adjustment can more accurately capture the fluctuation of the operator's running results, ensuring that the comparison threshold setting can adapt to the actual running environment. At the same time, it can reduce the impact of random errors caused by problems such as the scheduling of the computing core and the precision of floating-point numbers on the threshold setting.

[0019] In one possible implementation, when the accelerator card determines the difference between the first output result and the second output result, it can obtain the third output result corresponding to the first input data obtained by the processor running the operator to be detected. If the first output result and the third output result are consistent, the accelerator card determines the difference between the first output result and the second output result. If the first output result and the third output result are inconsistent, the minimum value between the difference value and the threshold of this round is used as the comparison threshold.

[0020] It should be understood that some operators may have multiple execution paths, and different execution paths may be used before and after the perturbation. Therefore, the output results of such operators before and after the perturbation are inherently erroneous. This application uses the output results obtained by the processor 200 running the operator to be detected as the true value to filter out the operators with multiple execution paths. Such operators do not need to be continuously decayed; they can be determined based on the difference value, thereby avoiding the misdetection of such operators as problematic operators when they are accurate.

[0021] In one possible implementation, each round of threshold adjustment in the multi-round threshold adjustment method includes: the accelerator card runs the operator to be detected multiple times to obtain M first output results and M difference values ​​between the M second output results; the accelerator card obtains M third output results obtained by the processor running the operator to be detected multiple times; if there is an inconsistency between the M third output results and the M first output results, the accelerator card determines the comparison threshold of the operator to be detected based on the M difference values; if all M third output results are consistent with the M first output results, the accelerator card determines the threshold for the next round based on the M difference values ​​and the threshold of the current round.

[0022] The above implementation method determines whether to proceed with the next round of adjustment based on the results obtained from multiple runs of the operator to be detected during each round of threshold adjustment. This reduces random errors caused by a single run, improves the reliability of each round of threshold adjustment, and ensures that the final comparison threshold can adapt to most detection environments, thus avoiding false detection problems.

[0023] In one possible implementation, when there are inconsistencies between the M third output results and the M first output results, the accelerator card determines the alignment threshold of the operator to be detected based on the M difference values, including: when all the M third output results are inconsistent with the M first output results, the accelerator card uses the minimum value among the M difference values ​​as the alignment threshold; when there are partial inconsistencies between the M third output results and the M first output results, the accelerator card divides the M difference values ​​into a consistent group and an inconsistent group, wherein at least one difference value in the consistent group corresponds to a third output result that is consistent with the first output result, and at least one difference value in the inconsistent group corresponds to a third output result that is inconsistent with the first output result; and the alignment threshold is determined based on the maximum value among at least one difference value in the consistent group and the minimum value among at least one difference value in the inconsistent group.

[0024] It should be understood that if all M first output results are consistent, it means that all M first output results are accurate. In this case, the threshold adjustment algorithm can be used to decay the threshold multiple times until the stopping condition is met. If all M first output results are inconsistent, it means that all M first output results are inaccurate. It is very likely that the operator executed different processing paths in two separate instances. In this case, threshold adjustment can be stopped. Since the M difference values ​​determined by the inaccurate first output results are also inaccurate, and these difference values ​​may be relatively large, the smallest difference value is selected from the M difference values ​​as the comparison threshold. When there is partial consistency and partial inconsistency, it indicates that some of the M first output results are accurate. This might mean the operator sometimes executes the same processing path twice, and sometimes it executes different processing paths. In this case, threshold adjustment can be stopped. The M difference values ​​are then divided into a consistent group and an inconsistent group. The third output result in the consistent group is consistent with the first output result, while the third output result in the inconsistent group is inconsistent with the first output result. The maximum difference U (the difference between the first and second output results) among multiple difference values ​​in the consistent group and the minimum difference V (the difference between multiple difference values ​​in the inconsistent group) are then determined. This determines the comparison threshold for the operator to be detected. The comparison threshold is greater than the maximum difference in the consistent group and less than the minimum difference in the inconsistent group. The comparison threshold can be any value in (U, V), or it can be determined from this interval by combining specific application scenarios and value selection methods, such as taking the median of the interval. This application does not impose specific limitations.

[0025] The above implementation method, by comparing the differences between M first output results and M third output results, can effectively identify operators that execute different computational paths before and after the disturbance, and provide corresponding threshold adjustment methods. This allows all operators to be detected to obtain their optimal comparison threshold. Using such comparison thresholds for operator detection can improve the accuracy of the detection results and avoid the problems of false detection and missed detection.

[0026] In one possible implementation, the accelerator card receives a detection instruction sent by the processor, and based on the detection instruction, obtains the first input data and the first output result of the operator to be detected. The calculation instruction and the detection instruction are generated by the processor based on a machine learning framework, which includes operator detection device code.

[0027] In simple terms, an accelerator card can include an operator detection device and a running device. The code of the operator detection device can be inserted into a machine learning framework, so that the machine learning framework can automatically run the operator detection device code when running the training script. When generating calculation instructions based on the training script, it can also generate detection instructions based on the operator detection device code. Then, the calculation instructions and detection instructions are sent to the accelerator card for execution. The running device executes the calculation instructions and runs each operator. The operator detection device executes the detection instructions and performs accuracy detection on the operators running on the accelerator card.

[0028] The aforementioned implementations, by inserting the operator detection device code into the machine learning framework, allow users to write computational scripts normally. While directly inserting the operator detection device code into the training script could also achieve operator detection during operation, this requires users to insert the code for each computational script, wasting manpower. This application inserts the operator detection device code into the machine learning framework, thus eliminating the need to modify the user's training script and automatically running the operator detection device to complete operator detection, reducing labor costs and improving the user experience.

[0029] Secondly, an operator detection device is provided, comprising: a receiving unit for receiving calculation instructions sent by a processor and running multiple operators based on the calculation instructions, wherein the calculation instructions are generated by the processor according to the training script of an artificial intelligence (AI) model; an acquisition unit for acquiring first input data and a first output result of the operator to be detected when the operation reaches the operator to be detected; a perturbation unit for perturbing the first input data to obtain second input data; a running unit for inputting the second input data into the first operator and running the first operator to obtain a second output result; and a comparison unit for comparing the first output result with the second output result to obtain an operator detection result.

[0030] The apparatus described in the second aspect monitors the progress of the training script. When the script reaches the operator to be detected, it acquires the first input data and the first output result of the operator to be detected. Then, it performs a slight perturbation on the first input data to obtain the second input data. The second input data is then input into the operator to be detected, and the operator to be detected is run again to obtain the second output result. By comparing the first output result before the perturbation and the second output result after the perturbation, the operator detection result of the accelerator card is obtained. This makes the operator detection of the accelerator card no longer dependent on the benchmark accelerator card, and the operator detection method is no longer limited. It can cover all possible detection scenarios and meet the user's detection needs.

[0031] In one possible implementation, an acquisition unit is used to monitor the running progress of multiple operators in an operator list. When the running reaches the operator to be detected, the unit acquires the first input data and the first output result of the operator to be detected. The operator list includes multiple floating-point operators and multiple hot-spot operators. The floating-point operators include operators used to process floating-point operations, and the hot-spot operators include operators with historical precision problems and operators expected to have precision problems in the future.

[0032] In one possible implementation, the comparison unit is used to obtain the comparison threshold of the operator to be detected based on the threshold list, compare the difference between the first output result and the second output result with the comparison threshold, and obtain the operator detection result. The operator detection result includes information about abnormal operators. If the difference value is greater than the threshold, the operator to be detected is determined to be an abnormal operator. The threshold list includes the comparison threshold corresponding to the operator. The comparison threshold is obtained by the accelerator card after attenuating the initial threshold through a multi-round threshold adjustment method before receiving the calculation instructions sent by the processor.

[0033] In one possible implementation, the device further includes a threshold adjustment unit, which is used to run the operator to be detected to obtain a first output result before perturbation and a second output result after perturbation. The threshold adjustment unit is used to determine the difference value between the first output result and the second output result. The threshold adjustment unit is used to determine the current round threshold as the comparison threshold of the operator to be detected when the comparison result between the difference value and the current round threshold reaches the stopping condition. The current round threshold includes the initial threshold or the threshold obtained by the previous round threshold adjustment. The threshold adjustment unit is used to attenuate the current round threshold to obtain the next round threshold when the comparison result between the difference value and the current round threshold does not reach the stopping condition.

[0034] In one possible implementation, the threshold adjustment unit is used to obtain the third output result corresponding to the first input data obtained by the processor running the operator to be detected; the threshold adjustment unit is used to determine the difference value between the first output result and the second output result when the first output result and the third output result are consistent; and the threshold adjustment unit is used to take the minimum value between the difference value and the threshold of the current round as the comparison threshold when the first output result and the third output result are inconsistent.

[0035] In one possible implementation, a threshold adjustment unit is used to run the operator to be detected multiple times to obtain M difference values ​​between M first output results and M second output results; a threshold adjustment unit is used to obtain M third output results obtained by the processor running the operator to be detected multiple times; a threshold adjustment unit is used to determine the comparison threshold of the operator to be detected based on the M difference values ​​when there is inconsistency between the M third output results and the M first output results; and a threshold adjustment unit is used to determine the threshold for the next round based on the M difference values ​​and the current round threshold when all M third output results are consistent with the M first output results.

[0036] In one possible implementation, a threshold adjustment unit is used to, when all M third output results are inconsistent with the M first output results, use the minimum value among the M difference values ​​as the comparison threshold. The threshold adjustment unit is also used to, when there are partial inconsistencies between the M third output results and the M first output results, divide the M difference values ​​into a consistent group and an inconsistent group, wherein at least one difference value in the consistent group corresponds to a third output result that is consistent with the first output result, and at least one difference value in the inconsistent group corresponds to a third output result that is inconsistent with the first output result. The threshold adjustment unit is used to determine the comparison threshold based on the maximum value of at least one difference value in the consistent group and the minimum value among at least one difference value in the inconsistent group.

[0037] In one possible implementation, a receiving unit is used to receive a detection instruction sent by the processor, and an acquisition unit is used to acquire the first input data and the first output result of the operator to be detected based on the detection instruction. The calculation instruction and the detection instruction are generated by the processor based on a machine learning framework, which includes operator detection device code.

[0038] Thirdly, a chip is provided, which includes a power supply circuit and a processing unit, wherein the power supply circuit supplies power to the multiple processing units and the processing units perform the operational steps of the method described in the first aspect.

[0039] Fourthly, a computing device is provided, comprising a processor and an accelerator card, the accelerator card performing the operational steps of the method described in the first aspect.

[0040] Fifthly, a computing device cluster is provided, the computing device cluster including at least one computing device, each computing device including a processor and a memory, the processor of the at least one computing device being configured to execute instructions stored in the memory of the at least one computing device, such that the computing device cluster implements the method described in the first aspect.

[0041] In a sixth aspect, a computer-readable storage medium is provided, wherein instructions are stored therein, and the instructions are executed by a computing device or a cluster of computing devices to implement the method described in the first aspect.

[0042] In a seventh aspect, a computer program product comprising instructions is provided, the computer program product including instructions capable of running on a computing device or stored in any available medium, and when the computer program product is run on a computing device or a cluster of computing devices, causing the computing device or cluster of computing devices to perform the method described in the first aspect.

[0043] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0044] Figure 1 This is an architecture diagram of an operator detection system provided in this application;

[0045] Figure 2 This is an example diagram of an operator detection system provided in this application deployed in a cloud environment;

[0046] Figure 3 This is a flowchart illustrating the steps of an operator detection method provided in this application during the configuration phase.

[0047] Figure 4 This is a flowchart illustrating the steps of a multi-round threshold adjustment method in an operator detection method provided in this application;

[0048] Figure 5 This is a flowchart illustrating the steps of an operator detection method provided in this application during the detection stage.

[0049] Figure 6 This is a schematic diagram of the structure of an operator detection device provided in this application;

[0050] Figure 7 This is a schematic diagram of the structure of a computing device provided in this application;

[0051] Figure 8 This is an example diagram of a computing device cluster provided in this application;

[0052] Figure 9 This is a schematic diagram of another computing device cluster provided in this application. Detailed Implementation

[0053] First, the application scenarios involved in this application will be explained.

[0054] Accelerator cards are hardware devices used to accelerate computing tasks and are widely used in fields such as AI and big data analytics. Accelerator cards typically include data processing units (DPUs), neural processing units (NPUs), graphics processing units (GPUs), extreme processing units (XPUs), and systems-on-chips (SoCs). Operators within an accelerator card are basic functional modules that perform specific computational operations, such as matrix multiplication and convolution. Since the accuracy of operators directly affects the accuracy of accelerator card calculations, operator problem detection is particularly important.

[0055] Typically, when testing the operator of an accelerator card under test, a benchmark accelerator card with verified operator accuracy is selected. The operator is then reproduced on the benchmark accelerator card, and the same test environment as the accelerator card under test is configured. This test environment includes hardware configuration, operating system, driver version, etc. The same input data and operators are then used to perform calculations on both the benchmark and the accelerator card under test. The calculation results are compared to determine whether the operator of the accelerator card under test meets the expected accuracy standard.

[0056] However, this operator detection method, which relies on benchmark accelerator cards, has many problems, which will be explained one by one below.

[0057] First, this operator-based detection method can only be used when the benchmark accelerator card's computational accuracy is higher than that of the accelerator card under test. If the benchmark accelerator card's performance is lower than that of the accelerator card under test, this method will not be usable. However, whether the benchmark accelerator card's performance is superior to that of the accelerator card under test is often unknown. If the accelerator card under test has higher performance, using the benchmark accelerator card's data as a benchmark for evaluation will distort the detection results for the accelerator card under test, causing the accelerator card under test, which should be considered accurate, to be misjudged as having insufficient performance.

[0058] Secondly, this operator detection method can only achieve ideal results for testing simple operators. Some complex operators are difficult to reproduce on benchmark accelerator cards. This is because the two accelerator cards may belong to different manufacturers or different models, and there may be differences in hardware configuration, operating system, and driver version. Reproducing complex operators is not only difficult to operate, but also increases development costs, making it impossible to detect operators in some complex operator scenarios.

[0059] Finally, this operator-based detection method requires that the testing environments of the two accelerator cards be consistent. Any slight difference can render the comparison results unreliable and lead to misjudgments. Therefore, the benchmark accelerator card to be tested needs to be similar to it. If a similar accelerator card cannot be found on the market, the accelerator card to be tested may not be able to be detected because the testing environment is difficult to ensure consistency, further limiting the detection scenarios.

[0060] In summary, current operator detection methods rely on benchmark accelerator cards, which limits their effectiveness, prevents them from covering all possible detection scenarios, and makes it difficult to meet users' detection needs.

[0061] To address the limitation of current operator detection methods, which rely on benchmark accelerator cards and thus restrict coverage of many detection scenarios, this application provides an operator detection method. This method monitors the progress of the training script on the accelerator card using an operator detection device. When the script reaches the operator to be detected, it acquires the first input data and the first output result. Then, it subtly perturbs the first input data to obtain second input data, which is then input into the operator to be detected. The operator is then re-run to obtain the second output result. By comparing the first output result before perturbation with the second output result after perturbation, the operator detection result of the accelerator card is obtained. This method eliminates the reliance on benchmark accelerator cards for operator detection, freeing the method from limitations and enabling coverage of all possible detection scenarios to meet user detection needs.

[0062] Figure 1 This is an architecture diagram of an operator detection system 1000 provided in this application, such as... Figure 1 As shown, the architecture includes an operator detection system 1000 and clients 2000, wherein the number of clients 2000 that establish communication connections with the operator detection system 1000 can be one or more. Figure 1 This application uses one client 2000 and one operator detection system 1000 as an example for illustration, and does not impose specific limitations. The client 2000 and the operator detection system 1000 can communicate via a wired network or a wireless network. Wired networks include Ethernet, overhead lines, cables, etc., while wireless networks include wireless local area networks (Wi-Fi), cellular networks, etc. They may also include the Internet, local area networks (LANs), etc., which simultaneously support wired and wireless connections, and of course, other network types may also be included, which are not specifically limited in this application.

[0063] Client 2000 is deployed on terminal devices or computing devices to enable human-computer interaction. Terminal devices include personal computers, smartphones, wearable devices, handheld processors, tablets, mobile laptops, augmented reality (AR) devices, virtual reality (VR) devices, smart conferencing devices, etc., without specific limitations. Computing devices can be bare metal servers (BMS), virtual machines, containers, or storage devices. BMS refers to a general-purpose physical server, such as an ARM server or an x86 server; a virtual machine refers to a complete computer system with full hardware system functionality simulated by software, running in a completely isolated environment. Any task that can be performed on a physical computer can also be performed in a virtual machine. When creating a virtual machine on a computing device, a portion of the physical machine's hard drive and memory capacity needs to be used as the virtual machine's hard drive and memory capacity. Each virtual machine has its own independent basic input / output system (BIOS), hard disk, and operating system, and can be operated like a physical machine. A container is a portable software unit that can combine an application and all its dependencies into a single software package. This package is not limited by the underlying host operating system, thus eliminating the need to build complex environments and simplifying the application development and deployment process.

[0064] The Operator Detection System 1000 can be deployed on computing devices, or on a cluster of computing devices. These devices can be bare metal servers (BMS), virtual machines, containers, or storage devices. A BMS refers to a general-purpose physical server, such as an ARM or x86 server. A virtual machine refers to a complete computer system simulated by software, possessing full hardware system functionality and running in a completely isolated environment. Any task that can be performed on a physical computer can also be performed in a virtual machine. When creating a virtual machine on a computing device, a portion of the physical machine's hard drive and memory capacity is used as the virtual machine's hard drive and memory capacity. Each virtual machine has an independent basic input / output system (BIOS), hard drive, and operating system, and can be operated like a physical machine. A container is a portable software unit that can combine an application and all its dependencies into a single software package. This package is not limited by the underlying host operating system, thus eliminating the need to build complex environments and simplifying the application development and deployment process.

[0065] Optionally, the client 2000 can be deployed on a terminal device, and the operator detection system can be deployed on a computing device or a cluster of computing devices. Alternatively, the client 2000 and the operator detection system 1000 can be deployed on the same computing device. Alternatively, the client 2000 and the operator detection system 1000 can be deployed on different computing devices within the same computing device cluster. Alternatively, the client 2000 and the operator detection system 1000 can be deployed on different computing devices within different computing device clusters; this application does not impose specific limitations.

[0066] Optionally, the client 2000 can be software or an application running on a user-controlled terminal device or computing device, such as a personal computer (PC) client, a web client accessed via a browser, an application (APP) client running on a mobile terminal, or a cloud platform console; this application does not impose specific limitations. Users holding the client 2000 can be developers training AI models, or testers checking the accuracy of accelerator card operators, etc.; this application does not impose specific limitations.

[0067] Optionally, the client 2000 can be a client specifically used to detect the accuracy of operators on accelerator cards, or it can be a machine learning platform, model service platform, or client running a machine learning framework that includes operator detection functionality. Such clients provide a series of tools and services for the entire process of defining, training, deploying, inferring, and managing machine learning models, such as TensorFlow, PyTorch, and Caffe. This application does not impose any specific limitations.

[0068] Optionally, client 2000 can also be a client of a cloud platform, such as the cloud platform's console. Client 2000 can also serve as a service interface for the operator detection system 1000 to provide services externally, allowing users to access the corresponding services by calling this service interface. The service interface can be at least one of the following forms: software development kit (SDK) or application programming interface (API).

[0069] Furthermore, the aforementioned client 2000 can provide users with operator detection cloud services, and users can obtain access to the operator detection system 1000 provided in this application by purchasing cloud services. Alternatively, the cloud platform can provide users with a comprehensive service, and the operator detection method provided in this application can be a sub-service within that comprehensive cloud service. For example, if the comprehensive service is an AI cloud service, the operator detection method in this application can be a sub-service within the AI ​​cloud service; this application does not impose any specific limitations.

[0070] The preceding text described in detail the possible deployment methods of the operator detection system 1000 and client 2000. In actual deployment, flexible deployment can be carried out according to specific application scenarios and business requirements. The following section provides examples of actual deployment methods for the operator detection system 1000 and client 2000 in specific application scenarios.

[0071] For example, suppose client 2000 and operator detection system 1000 are deployed on an enterprise's internal network. Operator detection system 1000 is located on a server or server cluster purchased by the enterprise. Client 2000 is a remote desktop connection tool or server management console software. Employees of the enterprise can remotely connect to the server or server cluster where operator detection system 1000 is located through client 2000 and use operator detection system 1000 to complete operator accuracy detection of accelerator cards in the server or server cluster.

[0072] For example, the operator detection system 1000 can be deployed in a cloud environment. Figure 2 This is an example diagram of an operator detection system 1000 provided in this application deployed in a cloud environment, such as... Figure 2 As shown, a user can initiate a purchase request for AI cloud services through client 2000. The AI ​​cloud services include an operator detection sub-service. After client 2000 sends the purchase request to the cloud platform, the cloud platform can grant client 2000 access to the operator detection system 1000. This allows the user to use the operator detection system 1000 provided in this application through client 2000 to perform operator detection on the accelerator card, or to perform routine and stable detection and diagnosis during the long-term development and maintenance of model training.

[0073] The cloud platform also maintains various basic resources, including computing resources, storage resources, network resources, and security resources, to meet the computing needs of the operator detection system 1000 under different scales and loads. Furthermore, these computing resources can be dynamically scaled according to the usage requirements of the operator detection system 1000 to ensure the stable operation of the operator detection system 1000 and provide users with reliable operator detection services.

[0074] It should be understood that the above application scenarios are for illustrative purposes only. The operator detection system 1000 and client 2000 can be flexibly deployed according to actual business needs. They will not be listed here.

[0075] Furthermore, the operator detection system 1000 may include an accelerator card 100 and a processor 200. The operator detection system 1000 is used to detect the operator execution accuracy of the accelerator card 100. The number of accelerator cards 100 and processors 200 in the operator detection system 1000 may be one or more. Figure 1 This application uses an accelerator card 100 and a processor 200 as examples for illustration, but does not impose any specific limitations.

[0076] The accelerator card 100 can be a graphics processing unit (GPU), a neural network processing unit (NPU), a data processing unit (DPU), etc., and this application does not impose a specific limitation. In some scenarios, the accelerator card is also referred to as an inference card. Optionally, the accelerator card 100 can also be a central processing unit (CPU), that is, the operator detection scheme of this application can also be used to test the accuracy of CPU operators, and this application does not impose a specific limitation.

[0077] The processor 200 can be any one or more of the following: CPU, graphics processing unit (GPU), microprocessor (MP), digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), complex programmable logic device (CPLD), etc., and this application does not impose a specific limitation. The processor 200 can also be an accelerator card, such as a GPU, NPU, DPU, etc., and this application does not impose a specific limitation.

[0078] Optionally, the accelerator card 100 and the processor 200 can establish a communication connection via a bus, such as a peripheral component interconnect express (PCIe) bus, an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), or a cache coherent interconnect for accelerators (CCIX). The unified bus is also known as the Lingqu bus.

[0079] Optionally, the accelerator card 100 and the processor 200 can also establish a communication connection through high-speed connection technology. In AI scenarios, there is a high demand for data transmission between the accelerator card and the processor, so communication between the processors can also be achieved through high-speed connection technology.

[0080] Among them, high-speed connection technology may include serial input / output (SIO). SIO is a data transmission interface that connects different functional modules or processing units inside the chip. In this case, the accelerator card 100 and the processor 200 can be packaged on a single chip, such as a dual-die co-packaged chip. Communication connections can be established between the two dies through SIO.

[0081] High-speed connectivity technologies may also include Huawei Cloud Communication Service (HCCS), a high-speed interconnect technology for cloud computing scenarios that provides high-speed, low-latency cloud communication services. HCCS enables direct connections between nodes while providing high bandwidth and reliability. It may also include NVLink technology, a high-speed interconnect technology for direct communication between GPUs or between GPUs and CPUs, suitable for high-performance computing and machine learning. NVLink can directly connect multiple GPUs and CPUs to achieve efficient transmission. It should be understood that the above examples are for illustrative purposes only and are not intended to limit the scope of this application.

[0082] Furthermore, the accelerator card 100 may include a running device 110 and an operator detection device 120, and the processor 200 may include a training script 210, a machine learning framework 220, and an operator list 230.

[0083] The Machine Learning Framework 220 is a software library or toolkit for building, training, and deploying machine learning models. It provides various application programming interfaces (APIs) and tools for users to build, train, and deploy models, including model definition, training, inference, optimization, and deployment. It can handle a wide range of machine learning tasks, from traditional statistical learning methods to modern deep learning techniques. Here, "user" refers to the user who trains the model, such as developers or programmers; this application does not specifically limit this definition.

[0084] For example, such as Figure 1 As shown, the machine learning framework 220 may include a model definition interface, allowing users to define the structure of various machine learning models, such as neural networks, large language models, decision trees, etc.; it may also include a loss function interface, allowing users to define loss functions; and it may also include an optimizer interface, supporting users to select optimization algorithms, such as gradient descent and adjusting hyperparameters. In specific implementations, the machine learning framework 220 may be TensorFlow, PyTorch, MindSpore, etc., and this application does not impose specific limitations.

[0085] Training script 210 is a piece of program code written by the user based on the aforementioned machine learning framework 220. Specifically, the machine learning framework 220 provides APIs and tools that users can use to write training scripts. Typically, training scripts may include defining a neural network model, loading a dataset, setting the loss function and optimizer, writing training steps, etc. They may also specify certain operations to be executed on the accelerator card 100, such as using specific syntax or notation to assign certain parts of the model to the accelerator card 100 for execution, so as to fully utilize the parallel computing capabilities of the accelerator card 100.

[0086] The execution device 110 receives computational instructions sent by the processor 200 based on the machine learning framework 220 and executes corresponding operators. These operators are various operations in the neural network, such as convolution, pooling, and matrix multiplication. These operations are executed in parallel on the accelerator card 100, improving computational efficiency and training speed. Specifically, the training script 210 is executed through the machine learning framework 220. The machine learning framework 220 can allocate appropriate computational tasks to the accelerator card 100 based on the content of the training script 210. At this time, it generates a series of computational instructions based on the operators included in the computational tasks, instructing the execution device 110 of the accelerator card 100 to execute the corresponding operators. The execution device 110 can execute each operator according to the execution order written in the training script based on these computational instructions. Figure 1 Operators 2, 3, and 4 are running in the process.

[0087] The operator detection device 120 is used to detect the accuracy of the operators run by the accelerator card 100. The code of the operator detection device 120 can be inserted into the machine learning framework 220, so that the machine learning framework can automatically run the operator detection device code when running the training script, and can also generate detection instructions based on the operator detection device code when generating calculation instructions based on the training script. Then, the calculation instructions and detection instructions are sent to the accelerator card 100 for execution. The running device 110 executes the calculation instructions and runs each operator, and the operator detection device executes the detection instructions to perform accuracy detection on the operators run by the accelerator card 100.

[0088] In practical implementation, the operator detection device code can be inserted into the machine learning framework 220. It should be understood that while directly inserting the operator detection device code into the training script could also achieve operator detection during operation, this would require the user to insert the operator detection device code for each computation script, wasting human resources. This application inserts the operator detection device code into the machine learning framework 220. This way, the user's training script does not need to be modified, and the operator detection device can automatically run to complete operator detection, reducing labor costs and improving the user experience.

[0089] The operator list 230 includes operator information for multiple operators to be detected. The operator list 230 is used by the operator detection device 120 to locate the operators to be detected. Any operator to be detected can be added to the operator list 230. The operator information may include the operator's identifier, such as the operator's ID or name; this application does not impose any limitations on this. The operators to be detected may include floating-point operations (FLOPs) and hotspot operators. FLOPs refer to operators performing floating-point calculations, typically basic mathematical operations such as addition, subtraction, multiplication, division, mathematical functions, reduction operations, matrix multiplication, broadcasting operations, and convolution operations. Hotspot operators refer to operators that have previously been detected as problematic. The aforementioned operators to be detected can be manually configured by the user or obtained based on default configurations; this application does not impose any specific limitations.

[0090] In this embodiment, the operator detection device 120 can monitor the running progress of the running device 110. When the running device 110 runs to the operator to be detected recorded in the operator list 230, the operator detection device 120 can obtain the first input data input to the operator to be detected and the first output result obtained after the operator to be detected runs. Then, the first input data is perturbed to obtain the second input data, and the second input data is input to the operator to be detected to obtain the second output result obtained after the operator to be detected runs repeatedly. Then, the first output result and the second output result are compared to obtain the comparison result, and the comparison result is sent to the processor 200 to generate the operator detection result, which can be a detection report.

[0091] For example, Figure 1 In the example, the operator list 230 includes operator 3. When the running device 110 runs to operator 3, the operator detection device 120 can obtain the input data and output result of operator 3, that is, the first input data and the first output result. Then, the first input data is perturbed to obtain the second input data. The second input data is then input into operator 3. Operator 3 is run repeatedly to obtain the second output result. The first output result and the second output result are then compared to obtain the comparison result of operator 3.

[0092] Similarly, operator list 230 also includes operator 8. When the running device 110 runs to operator 8, operator detection device 120 can obtain the input data and output result of operator 8, then perturb the input data, run operator 8 again, obtain the perturbed output result, and then compare the output result before and after perturbation to obtain the comparison result of operator 8. In this way, the comparison result of each operator in operator list 230 can be obtained, and then the comparison result is sent to processor 200 to generate a detection report for accelerator card 100.

[0093] Optionally, when perturbing the first input data x, the operator detection device 120 can use some perturbation algorithms to make the perturbed second input data x' close to the first input data x. This introduces some changes to the first input data x while maintaining its original distribution as much as possible. It should be understood that if the perturbation is too large, it may cause drastic changes in the operator's behavior, resulting in a significant difference between the two outputs, rendering the comparison meaningless. By using subtle perturbations, the basic behavior of the operator can be kept constant during the two calculations. By comparing the two outputs, if they are basically consistent, it can be inferred that the operator's precision is accurate; if the difference is large, it can be inferred that the operator's precision is incorrect. This comparison has higher reliability and effectiveness.

[0094] In specific implementations, the perturbation algorithm includes adding noise, such as randomly flipping elements in the first input data x; it also includes improving precision, such as converting the floating-point number of the first input data into a high-precision floating-point number; it also includes order swapping, such as swapping the order of the two addends in an addition operation; and it also includes repeated execution, that is, directly using the first input data as the second input data without modifying the first input data. The above examples are for illustration only and are not specifically limited in this application.

[0095] It should be understood that by subtly perturbing the input data of the operator to be tested, and then comparing the output result after perturbation with the output result before perturbation, the accuracy of the operator can be determined. This makes operator accuracy testing not dependent on benchmark accelerator cards, does not require users to purchase benchmark accelerator cards, and does not require users to reproduce the operator code on benchmark accelerator cards. Operator testing is low-cost and highly efficient.

[0096] Meanwhile, since it does not rely on benchmark accelerator cards, the limitation of detection scenarios can be solved. Accelerator cards with performance superior to benchmark accelerator cards can also perform operator accuracy testing, as can scenarios where operators are difficult to reproduce on benchmark accelerator cards, and scenarios where it is difficult to ensure consistency in the testing environment. This removes the limitations on operator accuracy testing and meets various user needs. Furthermore, this method does not require operator reproduction on benchmark accelerator cards, which not only reduces the hardware cost of benchmark accelerator cards but also reduces the manpower cost for developers.

[0097] Optionally, the accelerator card 100 may also store the comparison threshold corresponding to each operator. The comparison result may include the comparison result between the difference between the first output result and the second output result and the comparison threshold. It should be understood that different operators may obtain different results after perturbation. For example, some operators themselves will change significantly after slight perturbation, so the comparison threshold for these operators can be higher. Some operators will not change significantly after slight perturbation, so the comparison threshold for these operators can be lower. By pre-determining the mapping relationship between operators and comparison thresholds, comparisons can be performed based on the characteristics of each operator during comparison, resulting in more accurate detection results.

[0098] Optionally, when the running device 110 runs multiple operators, the operator detection device 120 will not interfere with the running device 110. Therefore, the operator detection device 120 can generate the comparison results of multiple operators in parallel or serially. The specific method can be determined according to the processing capability of the accelerator card 100, and this application does not make specific limitations.

[0099] It should be understood that traditional techniques rely on benchmark accelerator cards for detection, which can only complete operator detection before the model starts training. They cannot detect operator problems that occur after the model starts. However, in the technical solution of this application, the operator detection device 120 can detect operator accuracy problems online when the accelerator card 100 is running operators, so that operator accuracy problems can be detected in time and avoid the problems from escalating and affecting the accuracy of training results.

[0100] Meanwhile, large-scale training clusters using traditional techniques for operator detection require purchasing a large number of benchmark accelerator cards and reproducing a large number of operators, resulting in high detection costs. This application only requires inserting the code of the operator detection device into the training framework to achieve operator detection without interfering with the model training process. For large-scale training clusters, it can achieve online detection of operator accuracy at low cost and high efficiency.

[0101] In summary, the operator detection system provided in this application monitors the running progress of the training script on the accelerator card through an operator detection device. When the program reaches the operator to be detected, it obtains the first input data and the first output result of the operator to be detected. Then, it performs a slight perturbation on the first input data to obtain the second input data, and inputs the second input data into the operator to be detected. The operator to be detected is then run again to obtain the second output result. By comparing the first output result before the perturbation and the second output result after the perturbation, the operator detection result of the accelerator card is obtained. This makes the operator detection of the accelerator card no longer dependent on the benchmark accelerator card, and the operator detection method is no longer limited. It can cover all possible detection scenarios and meet the user's detection needs.

[0102] The above text combined Figure 1 and Figure 2 The operator detection system provided in this application has been explained and described below, in conjunction with... Figures 3-5 The operator detection method provided in this application is explained. This operator detection method may include a configuration phase and a detection phase. This application combines... Figures 3-4 The configuration phase was explained and described, combined with Figure 5 The testing phase was explained.

[0103] Figure 3 This is a flowchart illustrating the steps of an operator detection method provided in this application during the configuration phase. This method can be applied to, for example... Figure 1 In the operator detection system 1000 shown, such as Figure 3 As shown, the method may include the following steps:

[0104] S310: The processor 200 of the operator detection system 1000 inserts the operator detection device code into the machine learning framework.

[0105] In practice, a machine learning framework is a software library or toolkit used to build, train, and deploy machine learning models. Machine learning frameworks provide various APIs and tools for users to build, train, and deploy models. For a detailed explanation, please refer to [link / reference needed]. Figure 1 The description of the machine learning framework 220 in the embodiments will not be repeated here.

[0106] Alternatively, the operator detection device code can be inserted into the machine learning framework using decorators. Decorators are a way to modify the behavior of functions or methods using syntax features specific to programming languages. Decorators can dynamically add additional functionality to the original function without modifying its definition.

[0107] In this embodiment, the operator detection device is used to detect the accuracy of the accelerator card when running operators. After the operator detection device code is inserted into the machine learning framework, the machine learning framework of the processor 200 can automatically run the operator detection device code when running the training script, sending instructions to the accelerator card 100. This allows the accelerator card 100 to monitor the running progress of the training script when running operators. When it reaches the operator to be detected, it obtains the first input data and the first output result of the operator to be detected, then performs a slight perturbation on the first input data to obtain the second input data, and then inputs the second input data into the operator to be detected, re-running the operator to obtain the second output result. By comparing the first output result before the perturbation and the second output result after the perturbation, the operator detection result of the accelerator card is obtained. This makes the operator detection of the accelerator card no longer dependent on the benchmark accelerator card, and the operator detection method is no longer limited, covering all possible detection scenarios and meeting the user's detection needs.

[0108] It should be understood that while directly inserting the operator detection device code into the training script could achieve operator detection during operation, this would require the user to insert the operator detection device code for each computation script, wasting human resources. This application inserts the operator detection device code into the machine learning framework 220 via a decorator. This way, the user's training script does not need to be modified, and the operator detection device can automatically run to complete operator detection, reducing labor costs and improving the user experience.

[0109] In a specific implementation, the operator detection system 1000 can execute S310 when the machine learning framework starts, integrating the operator detection device with the machine learning framework. Alternatively, the operator detection system 1000 can execute S310 upon receiving a user's configuration request. The user can send a configuration request through the client 2000 when the operator detection device is needed, inserting the device's code into the machine learning framework. This allows the user to perform on-demand detection, reducing system overhead. Alternatively, the operator detection system 1000 can also automatically execute S310 upon receiving a training script sent by the user, meaning S310 occurs after S320. The above examples illustrate possible conditions for the operator detection system 1000 to trigger the execution of S310; other conditions may also trigger the execution of S310, which this application does not specifically limit.

[0110] S320: Client 2000 sends training script to processor 200 of operator detection system 1000.

[0111] In practice, a training script is a piece of code written by the user based on the machine learning framework to train a machine learning model. Specifically, the machine learning framework provides APIs and tools that users can use to write training scripts. When the training script runs, it also calls these APIs or tools to achieve the desired functionality. Typically, a training script may include defining a neural network model, loading a dataset, setting the loss function and optimizer, and writing training steps, etc. For details, please refer to [reference needed]. Figure 1 The relevant descriptions of training script 210 in the embodiments will not be repeated here.

[0112] S330: The processor 200 of the operator detection system 1000 determines at least one operator to be detected and generates an operator list.

[0113] In specific implementation, the operator list may include the identifiers of the operators to be detected, such as the identifier (ID) and name of the operator to be detected, and may also include other required information, which is not specifically limited in this application. The operator to be detected refers to the operator that needs to be detected by the operator detection system 1000. The operator to be detected may include floating-point operations (FLOPs) and hotspot operators. Floating-point operators refer to operators that perform floating-point calculations, typically basic mathematical operations such as addition, subtraction, multiplication, division, mathematical functions, reduction operations, matrix multiplication, broadcasting operations, and convolution operations. Hotspot operators refer to operators that have previously been detected as problematic. The above-mentioned operators to be detected can be manually configured by the user or obtained based on default configurations, which is not specifically limited in this application.

[0114] Optionally, the operator to be detected may include user-configured operators. Specifically, the user can input the operator to be detected through the client 2000, and the client 2000 sends a configuration request carrying the operator to be detected to the operator detection system 1000. The processor 200 of the operator detection system 1000 can add the operator to be detected in the configuration request to the operator list. Typically, the user-configured operators are hotspot operators. Users can add some operators that have previously been detected as problematic, as well as some operators that may cause problems, based on experience. This application does not impose specific limitations on this.

[0115] Optionally, the operator to be detected may include operators configured by default in the system, such as some commonly used floating-point operators, which can be used as the default operators to be detected. If the user does not need to configure the operator to be detected, S330 may not need to execute it and can directly use the system's default operator list. This application does not impose specific limitations.

[0116] Optionally, the operator to be detected can also be the operator determined by the processor 200 based on the user's training script. Specifically, the training script can be read and parsed to extract relevant information for all operators. This step can be implemented using the APIs of machine learning frameworks, such as TensorFlow's tf.keras.layers and PyTorch's torch.nn. For floating-point operators, whether they are floating-point operators can be determined based on their name or type. This step can be implemented using the APIs provided by machine learning frameworks; for example, the model summary function of TensorFlow and PyTorch can list all operators and their types in the model. For hotspot operators, operators that have historically encountered problems can be identified as hotspot operators based on the historical training logs and error reports of the accelerator card 100. Furthermore, potential hotspot operators can be identified based on performance data during training, such as operator execution time and memory usage.

[0117] The above examples illustrate possible methods for identifying floating-point operators and hotspot operators. Of course, other methods can also be used to identify the operators to be detected, and this application does not impose any specific limitations.

[0118] Optionally, the operator list is used by the operator detection device 120 to monitor the running progress of operators. When the device reaches an operator to be detected recorded in the operator list, the operator detection device 120 will acquire the first input data and the first output result of that operator. This process can be implemented through hook operations, which are mechanisms for dynamically inserting custom code to monitor and manipulate operator behavior in real time within a machine learning framework.

[0119] Specifically, after determining the list of operators, the processor 200 can register a hook function on the operator to be detected in the list. This hook function is used to obtain the first input data and the first output result of the operator. This step can also be implemented through the API of the machine learning framework (such as register_forward_hook). In this way, when the operator to be detected runs, the framework will automatically call the hook function to obtain the first input data and the first output result. By implementing the above process through the hook function, the computation script can be modified without modifying it. Instead, additional functions can be dynamically implemented during the training script runtime through the framework's internal mechanism, enabling monitoring and debugging.

[0120] It's important to note that machine learning involves forward computation and backpropagation. Backpropagation is automatically implemented by the machine learning framework. The secondary output obtained through hook-based methods often differs slightly from the original result of the actual operation in the network. However, this difference is not due to operator precision issues. Therefore, gradient reconstruction can mitigate this inherent difference and prevent false detections. Consequently, hook functions need to be differentiated between forward and backward hook functions. This allows the operator detection device to promptly determine whether the operator to be detected is a forward or backward operator during operation, and thus execute different processes accordingly.

[0121] S340: The processor 200 of the operator detection system 1000 determines the comparison threshold corresponding to each operator to be detected and obtains a threshold list.

[0122] In practice, the threshold list includes the comparison threshold for each operator to be detected. It should be understood that different operators may yield different results after perturbation. For example, some operators themselves change significantly with slight perturbation, so their comparison threshold can be higher; conversely, some operators do not change significantly with slight perturbation, so their comparison threshold can be lower. By pre-determining the mapping relationship between operators and comparison thresholds, comparisons can be performed based on the characteristics of each operator, resulting in more accurate detection results.

[0123] In one possible implementation, the comparison thresholds in the threshold list can be fixed thresholds preset by the system or fixed thresholds set by the user based on experience. If set by the user, the user can input the mapping relationship between the comparison thresholds to be set and the operators to be detected through the client 2000. The client 2000 sends the mapping relationship to the operator detection system 1000, and the operator detection system 1000 updates the threshold list accordingly. Each training script can use the threshold list to complete operator detection.

[0124] In another possible implementation, the alignment thresholds in the threshold list can be dynamically obtained based on a threshold adjustment algorithm. Each training script can use this algorithm to determine the thresholds for its operators before execution. Different training scripts may obtain the same or different thresholds using this method. This allows for more accurate determination of the alignment thresholds for the operators to be detected based on different training scripts, resulting in higher detection precision.

[0125] Optionally, an initial threshold can be assigned to the operator to be detected, and then multiple rounds of threshold adjustment can be performed. In each round, the first output result before the operator is perturbed and the second output result after the perturbed are calculated, the difference between the two is determined, and the difference is compared with the threshold of the current round. If the comparison result reaches the stopping condition, the threshold of the current round is used as the comparison threshold of the operator to be detected. If the comparison result does not reach the stopping condition, the threshold of the current round is decayed to obtain the threshold of the next round, and then the process is repeated until the comparison threshold corresponding to the operator to be detected is obtained.

[0126] In practice, the threshold for the next round can be a decay value calculated using a decay formula, such as half the threshold for the current round. Alternatively, the threshold for the next round can be the difference value from the current round. Or, the threshold for the next round can be obtained based on both the decay value and the difference value, for example, the maximum of the decay value and the difference value can be used as the threshold for the next round.

[0127] In specific implementation, the stopping conditions may include the threshold value in this round being less than the difference value, or the number of threshold adjustment rounds reaching a preset upper limit value, or the threshold value in this round reaching a preset minimum value, or other stopping conditions, or a combination of the above stopping conditions. This application does not make specific limitations.

[0128] For example, suppose the operator to be detected is operator X, the initial threshold assigned to operator X is Q1, and the threshold for the next round is the maximum of the decay value and the difference value. The stopping condition is that the threshold for this round is less than the difference value, and the decay formula is Q. n+1 =0.5Q n That is, it decays by half each time.

[0129] The first round of threshold adjustment begins by obtaining the first input data Y of operator X. 11 And the first output result Z 11 The first input data Y1 is perturbed to obtain the second input data Y. 12 Rerun operator X to obtain the second input data Y. 12 The corresponding second output result Z 12 Obtain the first output result Z before the perturbation. 11 And the second output result Z after perturbation 12The difference value P1 between the two is compared with the threshold Q1 of the current round. If the threshold Q1 of the current round is greater than the difference value P1, then the threshold comparison of the next round can be carried out. When determining the threshold of the next round, if the decay value is 0.5Q1 and the difference value P1 is less than the decay value Q1, then the decay value of the next round is Q2 = 0.5Q1.

[0130] The second round of threshold adjustment begins, obtaining the first input data Y of operator X. 21 And the first output result Z 21 For the first input data Y 21 Perturbation is performed to obtain the second input data Y 22 Rerun operator X to obtain the second input data Y. 22 The corresponding second output result Z 22 Obtain the first output result Z before the perturbation. 21 And the second output result Z after perturbation 22 The difference value P2 is compared with the current threshold Q2. If the current threshold Q2 is less than or equal to the difference value P2, the threshold adjustment can stop, and the current threshold Q2 is used as the comparison threshold for operator X. Of course, if the current threshold Q2 is greater than the difference value P2, the next round of threshold comparison can be performed. When determining the threshold for the next round, the decay value is 0.5Q2. If the difference value P2 is less than the decay value Q2, then the decay value Q3 for the next round is 0.5Q2. If the difference value P2 is not less than the decay value Q2, then the decay value Q3 for the next round is P2, and so on, performing multiple rounds of threshold determination until the stopping condition is met. These steps will not be elaborated upon here.

[0131] It should be understood that although operators perform basic addition, subtraction, multiplication, and division operations, the results of each repeated run of the operator may differ during actual computation. The reasons for these differences are complex and varied. For example, on an accelerator card, multiple computing cores may execute tasks in parallel, and the scheduling order and computational parallelism of these cores may lead to slight variations in the results of each run. Furthermore, operators on accelerator cards typically use floating-point arithmetic, which is essentially an approximation. Due to the precision limitations of floating-point numbers, many addition, subtraction, multiplication, and division results may differ. Additionally, the hardware of the accelerator card may optimize floating-point operations, such as by using different rounding modes to improve performance, which can also cause slight variations in the results of each operation. Of course, there are other possible reasons, which will not be listed here. Therefore, by adjusting the threshold as described above, running the operator to be tested multiple times to determine its threshold before the user begins training the model can improve the stability and accuracy of the final determined threshold.

[0132] Optionally, during each round of threshold adjustment, the processor 200 can also run the operator to be detected to obtain the third output result corresponding to the first input data, and compare it with the first output result as the true value. If they match, it means that the first output result is accurate. Then, as described above, multiple rounds of attenuation are performed based on the first output result before perturbation and the second output result after perturbation to determine the comparison threshold of the operator to be detected. If they do not match, it means that the first output result is inaccurate. At this time, the threshold adjustment can be stopped, and the comparison threshold can be determined based on the difference value (the difference between the second output result and the first output result). That is to say, the stopping condition can also include whether the third output result of the processor 200 matches the first output result.

[0133] It should be understood that some operators may have multiple execution paths, and different execution paths may be used before and after the perturbation. Therefore, the output results of such operators before and after the perturbation are inherently erroneous. This application uses the output results obtained by the processor 200 running the operator to be detected as the true value to filter out the operators with multiple execution paths. Such operators do not need to be continuously decayed; they can be determined based on the difference value, thereby avoiding the misdetection of such operators as problematic operators when they are accurate.

[0134] Furthermore, during each round of threshold adjustment, the processor 200 and accelerator card 100 can run the operator to be detected multiple times to obtain multiple truth value comparison results between multiple first output results and multiple third output results before the perturbation. Based on these multiple truth value comparison results, it is determined whether the multiple first output results are accurate. If inaccurate, the threshold adjustment stops. If accurate, the accelerator card 100 can run the operator to be detected multiple times to obtain multiple second output results after the perturbation. Then, the multiple first output results and multiple second output results are compared to obtain multiple comparison results. Based on these comparison results, it is determined whether the next round of threshold adjustment is needed. If not, the threshold adjustment stops; if so, the threshold is attenuated, and then the next round of threshold adjustment is performed, and so on. It should be understood that determining whether to stop threshold adjustment based on multiple comparison results during each round of threshold adjustment can improve the accuracy of threshold adjustment, reduce the impact of a single outlier, and thus improve the reliability of the overall decision.

[0135] For example, Figure 4 This is a flowchart illustrating the steps of a multi-round threshold adjustment method in an operator detection method provided in this application, as shown below. Figure 4As shown, during each round of threshold adjustment, firstly, the processor 200 runs M times based on the first input data to obtain M third output results, the accelerator card 100 runs M times based on the first input data to obtain M first output results, the first input data is perturbed to obtain second input data, and the process runs M times based on the second input data to obtain M second output results. Finally, M difference values ​​are determined between the M first output results and the M second output results.

[0136] Next, the M third output results generated by the processor 200 are used as true values ​​to determine whether the M first output results are accurate. Specifically, this is determined by comparing each first output result with the third output result to see if they are consistent. There are three types of comparison results: all consistent, all inconsistent, and partially consistent. The subsequent processing flow for each of the three types of comparison results will be explained below.

[0137] If the comparison results are all inconsistent, it means that all M first output results are inaccurate. It is very likely that the operator executed different processing paths in two separate instances. Therefore, the threshold adjustment can be stopped. Since the M difference values ​​determined by the inaccurate first output results are also inaccurate, and these difference values ​​may be relatively large, the smallest difference value is selected from the M difference values ​​as the comparison threshold.

[0138] When the comparison results are partially consistent and partially inconsistent, it indicates that some of the M first output results are accurate. This may mean that the operator sometimes executes the same processing path twice, and sometimes it executes different processing paths. In this case, threshold adjustment can be stopped. Then, the M difference values ​​are divided into consistent and inconsistent groups. The third output result in the consistent group is consistent with the first output result, while the third output result in the inconsistent group is inconsistent with the first output result. Then, the maximum difference U (the difference between the first and second output results) among multiple difference values ​​in the consistent group and the minimum difference V (the difference between multiple difference values ​​in the inconsistent group) are determined. This determines the comparison threshold for the operator to be detected. The comparison threshold is greater than the maximum difference in the consistent group and less than the minimum difference in the inconsistent group. The comparison threshold can be any value in (U, V), or it can be determined from this interval by combining specific application scenarios and some value selection methods, such as taking the median of the interval. This application does not impose specific limitations.

[0139] If all comparison results are identical, it means that all M first output results are accurate. Therefore, the threshold adjustment algorithm described above can be used to decay the threshold multiple times until the stopping condition is met. Specifically, first determine if the current round's threshold is greater than the M difference values. If the current round's threshold is less than or equal to the M difference values, threshold adjustment can be stopped, and the largest difference value among the M difference values ​​is used as the comparison threshold. If the current round's threshold is greater than the M difference values, the threshold for the next round is calculated using the decay formula. For example, the next round's threshold might be half of the current round's threshold. Then, the threshold is adjusted according to... Figure 4 The process shown redefines M first output results, M second output results, M third output results, and so on.

[0140] It should be noted that each third, first, and second output result corresponds to a running order identifier. For example, the third, first, and second output results obtained in the first run correspond to identifier 1, the third, first, and second output results obtained in the second run correspond to identifier 2, and so on. Thus, when comparing M third output results and M first output results, the third output results with the same order identifier are compared with the first data results. When determining the M differences, the first and second output results with the same order identifier are also compared.

[0141] It should be noted that, Figure 4 For ease of explanation, only some of the stopping conditions for threshold adjustment are shown. In actual implementation, more stopping conditions can be added, such as whether the threshold has reached the preset minimum value in this round, or whether the number of adjustment rounds has reached the upper limit, or other stopping conditions, or a combination of multiple stopping conditions. This application does not specify the specifics.

[0142] It should be understood that each operator to be detected can be configured according to... Figure 4 The method shown initiates multiple rounds of threshold adjustment until the threshold stabilizes or an error is detected based on the processor's third output, thereby obtaining the optimal comparison threshold for each operator to be detected and generating a threshold list. Operator detection is then performed based on this threshold list, resulting in more accurate detection results.

[0143] It should be noted that the threshold list can be generated by the processor 200 configuration, but the operations of determining whether the third output result is consistent with the first data result, determining the difference value, and determining whether to stop threshold adjustment can be implemented by the processor or by the accelerator card. This application does not make any specific limitation.

[0144] S350: The processor 200 of the operator detection system 1000 sends the operator list and threshold list to the accelerator card 100.

[0145] Specifically, the accelerator card 100 can store the operator list and threshold list. During the training script execution, the accelerator card 100 can receive calculation instructions and detection instructions sent by the processor 200. The calculation instructions are used to run various operators in the training script, and the detection instructions are used to detect the running progress of the training script according to the operator list. When the script reaches the operator to be detected contained in the operator list, the first input data and the first output data of the operator are obtained. Then, the first input data is perturbed to obtain the second input data. The operator is then run again to obtain the second output data corresponding to the second input data. Then, according to the comparison threshold corresponding to the operator in the threshold list, the difference value between the first output result and the second output result is obtained. The difference value is compared with the comparison threshold to obtain the comparison result. The comparison result is sent to the processor to obtain the detection report.

[0146] S360: The processor 200 of the operator detection system 1000 sends a configuration completion message to the client 2000.

[0147] It should be understood that, through the steps S310-S360 described above, after the user uploads the training script, they can wait for the operator detection system 1000 to complete its configuration automatically. After receiving the configuration completion message from the operator detection system 1000, the user can then begin the detection phase and start testing the operator accuracy of the accelerator card 100. The entire configuration process is simple, fast, and efficient, improving the user experience. Of course, users can also perform some custom configurations according to their needs, such as configuring the operator list and threshold list, etc. For details, please refer to the foregoing content; this application does not impose specific limitations.

[0148] The above text passed Figure 3 and Figure 4 The steps of the operator detection method provided in this application during the configuration phase are described in detail below. Figure 5 The steps and procedures of the operator detection method provided in this application during the detection stage are explained.

[0149] Figure 5 This is a flowchart illustrating the steps of an operator detection method provided in this application during the detection phase, as shown below. Figure 5 As shown, the method may include the following steps:

[0150] S510: Client 2000 sends a script execution request to processor 200 of operator detection system 1000.

[0151] In its implementation, the script execution request can carry the identifier of the training script to be executed, such as the script name and script ID, so that the processor 200 can determine which training script needs to be executed. It should be understood that the technical solution of this application can detect operator accuracy online during the execution of the training script, and promptly identify and provide abnormal operators to the user, preventing operators with accuracy problems from accumulating errors as the training script runs, thus affecting the model training effect. Therefore, when a user initiates a script execution request during script training, the operator detection system 1000 will automatically begin the operator detection process.

[0152] S520: The processor 200 of the operator detection system 1000 sends calculation instructions and detection instructions to the accelerator card 100.

[0153] In specific implementation, the processor of the operator detection system 1000 can determine the training script to be run based on the identifier in the script execution request, and execute the training script based on the machine learning framework 220. The machine learning framework 220 can generate appropriate computing tasks based on the content of the training script 210 and allocate them to the accelerator card 100 for execution. At this time, a series of computing instructions will be generated based on the operators contained in the computing tasks, and the execution device 110 of the accelerator card 100 will be notified to run the corresponding operators. The execution device 110 can execute each operator according to the execution order written in the training script based on the computing instructions, for example... Figure 1 Operators 2, 3, and 4 are running in the process.

[0154] As can be seen from the foregoing, the code of the operator detection device 120 is inserted into the machine learning framework 220. Therefore, when the machine learning framework 220 generates corresponding calculation instructions based on the training script, it can automatically run the operator detection device code, generate detection instructions, and then send the calculation instructions and detection instructions to the accelerator card 100 for execution. The running device 110 of the accelerator card 100 executes the calculation instructions and runs the corresponding operators. The operator detection device 120 executes the detection instructions and performs accuracy detection on the operators executed by the running device 110.

[0155] S530: The accelerator card 100 of the operator detection system 1000 runs multiple operators and obtains the first input data and first output result of the operator to be detected based on the operator list.

[0156] In specific implementation, the running device 110 of the accelerator card 100 executes the calculation instruction and runs multiple operators. The operator detection device 120 can execute the detection instruction to monitor the running progress of the running device 110. When running to the operator to be detected in the operator list, the first input data and the first output result of the operator to be detected are obtained.

[0157] As can be seen from the foregoing, during the configuration phase, the operator detection system 1000 registers the hook function on the operator to be detected in the operator list according to the operator list. The hook function is used to obtain the first input data and the first output result of the operator. Therefore, the operator detection device 120 can automatically call the hook function to obtain the first input data and the first output result when the operator to be detected is running.

[0158] It should be noted that the hook function includes forward hook functions and backward hook functions. This is because the backpropagation operator still needs to undergo gradient reconstruction. For details, please refer to the previous content, which will not be repeated here. In this way, the operator detection device can promptly determine whether the operator to be detected is a forward operator or a backward operator when the operator to be detected is running, and thus execute the corresponding different processes according to the two different operators.

[0159] S540: The accelerator card 100 of the operator detection system 1000 perturbs the first input data to obtain the second input data.

[0160] In practical implementation, when perturbing the first input data, some perturbation algorithms can be used to make the perturbed second input data x' close to the first input data x. This introduces some changes to the first input data x while maintaining its original distribution as much as possible. It should be understood that if the perturbation is too large, it may cause drastic changes in the operator's behavior, resulting in a significant difference between the two outputs, rendering the comparison meaningless. By using subtle perturbations, the basic behavior of the operator can be kept constant during the two calculations. Comparing the two outputs in this way, if they are basically consistent, it can be inferred that the operator's precision is accurate; if they differ significantly, it can be inferred that the operator's precision is incorrect. This comparison has higher reliability and effectiveness.

[0161] Optionally, the perturbation algorithm may include precision enhancement, which refers to increasing the precision of the numerical value from a lower level to a higher level, such as upgrading from 16-bit floating-point (FP16) to 32-bit floating-point (FP32). Specifically, it may be to improve the precision of forward computation or backward propagation, and this application does not make specific limitations. It should be understood that by perturbing by modifying the precision, the perturbed second input data can be made closer to the first input data.

[0162] Optionally, the perturbation algorithm may include adding noise, which refers to modifying the bits of the data through bit operations, such as modifying the last bit. For example, for a data instance of float16: 0001010101111000, flipping the last bit of the data yields the data 0001010101111001. The error between the two is small. By perturbing the first input data in this way, the perturbed second input data can be made closer to the first input data.

[0163] Optionally, the perturbation algorithm may include order swapping, which refers to changing the execution order of certain operators. For example, changing the execution order of addition operators. Because floating-point numbers have the problem of large numbers consuming small numbers, and floating-point addition calculations do not satisfy the associative law, changing the execution order of addition can lead to different final results in large-scale numerical operations, such as large matrix operations or long-order accumulation. By perturbing the first input data in this way, the obtained second input data will have a small difference from the first input data.

[0164] Optionally, the perturbation algorithm may also include repeated execution, which means that the first input data is not perturbed and is directly used as the second input data. As can be seen from the above, many reasons can cause differences in the results of repeated operation of the operator, such as differences caused by the precision of floating-point operations, differences caused by the parallel scheduling order, etc. Therefore, directly using the first input data as the second input data can also achieve operator detection.

[0165] Optionally, the perturbation algorithm also includes equivalent substitution, which refers to using other versions of the operator. For example, if the operator version in the training script is V1, then obtain the V2 version of the operator, input the first input data into the V2 version of the operator, and obtain the second output data.

[0166] It should be understood that the above examples provide some possible perturbation algorithms, and other perturbation algorithms can certainly be used; this application does not impose any specific limitations. In specific implementations, the perturbation algorithm can be configured by the user during the configuration phase or it can be the system default; this application does not impose any specific limitations.

[0167] S550: The accelerator card 100 of the operator detection system 1000 inputs the second input data into the operator to be detected, runs the operator to be detected, and obtains the second output result.

[0168] In a specific implementation, the operator detection system 1000 can rerun the operator to be detected based on the capabilities of the machine learning framework 220. The machine learning framework 220 usually provides an API for secondary execution of operators, through which the above steps can be achieved.

[0169] Optionally, if the operator to be detected is an operator used in the backpropagation process, gradient reconstruction is also required during the second run to reconstruct the gradient used during the first run, ensuring gradient consistency during the second run, and thus ensuring that the operator behavior during the two runs is approximately the same. Only in this way can the comparison result and the detection report obtained be accurate and reliable. Among them, whether the operator is an operator for backpropagation can be determined based on a hook function in S530. In specific implementation, gradient reconstruction can be achieved through methods such as numerical differentiation, automatic differentiation, analytical differentiation, and adjoint method, etc., which are not specifically limited in this application.

[0170] S560: The acceleration card 100 of the operator detection system 1000 obtains the comparison threshold of the operator to be detected based on the threshold list, and obtains the comparison result according to the first output result, the second output result, and the comparison threshold.

[0171] Optionally, the operator detection system 1000 can obtain the comparison threshold of the operator to be detected, and then compare the difference value between the first input result and the second output result with the comparison threshold to obtain the comparison result. Determine whether the difference value is greater than the comparison threshold. If it is greater than the comparison threshold, it is determined that the operator does not meet the standard. If it is not greater than the comparison threshold, it means that the accuracy of the operator is within the normal range.

[0172] Optionally, the operator detection system 1000 can also determine whether the difference value is greater than the comparison threshold based on the comparison accuracy standard. It should be understood that there is a rounding error when the acceleration card processes floating-point numbers. In theory, two equal numerical values may also be wrongly determined to be unequal due to the rounding error. By setting the comparison accuracy standard, the misjudgment caused by the rounding error can be avoided.

[0173] In specific implementation, the comparison accuracy standard can be set by the user, or can be the system default setting, or can also be set according to the user's selection after the system provides some options of the comparison accuracy standard to the user, which are not specifically limited in this application.

[0174] Optionally, the comparison accuracy standard may include an equality standard: determining whether numerical values are equal based on the minimum representation accuracy of the numerical values in the acceleration card. Specifically, assume that A and B are two real numbers, and E is the minimum representation accuracy of the current data type of the acceleration card 100. Then when |A - B| < E, A and B are equal. For example, in the float32 representation accuracy, the accuracy of the floating-point number is about 6 - 7 decimal digits, that is, the number of significant digits of the single-precision floating-point number is about 7 digits. For the real numbers 1 and 1.00000001, the two can be determined to be equal. The above example is for illustration and is not specifically limited in this application.

[0175] Optionally, the comparison accuracy standard may include a first standard: whether the relative error between the detected calculation result and the scientific calculation result exceeds a given threshold; if it does, the operator is deemed substandard. Here, the scientific calculation result refers to the true value obtained by the processor running the operator. The given threshold can be a pre-set threshold, and this standard can be used when third-party scientific calculation results are available; otherwise, other comparison accuracy standards can be used.

[0176] Optionally, the comparison accuracy standard may include a second standard: based on the ratio of the first output result y1 to the second output result y2 and the comparison threshold e, the formula of the second standard may be as follows (1). When the formula of the second standard is satisfied, the operator is determined to be substandard or abnormal.

[0177]

[0178] Optionally, the comparison accuracy may include a third standard: the second output result y2 is corrected by the gradient value, and the ratio of the corrected second output result to the first output result is compared with the threshold e. The formula of the third standard can be as follows (2). When the formula of the third standard is satisfied, the operator is judged to be substandard or abnormal.

[0179]

[0180] In the specific implementation, the equality criterion is used to judge whether the difference value is equal to the comparison threshold, and the first, second, and third criteria are used to judge whether the operator meets the criteria. The first criterion requires the existence of a CPU truth value. If a CPU truth value is not available, the second or third criterion can be used. However, the third criterion requires gradient calculation, which incurs significant overhead. Therefore, in scenarios with high latency requirements, the second criterion can be used, while in scenarios with low latency requirements, the third criterion can be used. It should be understood that the above criteria are for illustrative purposes only. In specific implementations, more criteria can be set according to the actual business scenario, and this application does not impose specific limitations.

[0181] S570: The accelerator card 100 of the operator detection system 1000 sends the comparison results to the processor 200.

[0182] It should be understood that since some gradient calculations are also required when comparing the difference value with the threshold, the comparison needs to be completed on the accelerator card 100 before the comparison result is sent to the processor 200 to generate a detection report.

[0183] S580: The processor 200 of the operator detection system 1000 generates a detection report based on the comparison results.

[0184] In practice, the detection report should at least include identifiers of operators deemed substandard, such as operator ID and operator name, and may also include other content, which can be determined according to user needs. The more detailed the detection report content, the greater the export overhead and the lower the export efficiency; the simpler the detection report content, the lower the overhead and the higher the efficiency. Users can pre-configure the export level during the configuration phase, and the processor 200 on the S580 can generate the corresponding detection report based on the export level and comparison results.

[0185] For example, at the first export level, the detection report may include basic information about the operator and the first input data, such as the operator name or operator ID, and the type, shape, and value range of the first input data. At the second export level, the detection report may also include distribution data of the difference between the first output result and the second output result, such as generating a corresponding distribution histogram for user analysis, and may also include information such as the call stack of the abnormal operator function. At the third export level, the detection report may also include the original data of the first input data, the first output result, the second input data, and the second output result. It should be understood that the above export level classification is for illustrative purposes only and is not intended to be specific.

[0186] S590: The processor 200 of the operator detection system 1000 sends a detection report to the client 2000.

[0187] Optionally, if the perturbation algorithm used by S540 is to increase the precision, that is, the second output result is the calculation result of increasing the precision of the operator to be detected, then the second output result can replace the first output result, and the subsequent operation steps on the training script can be executed to achieve the precision repair of the model.

[0188] As described above, the technical solution provided in this application allows users to train a model while simultaneously automatically detecting the operator precision on the accelerator card 100 simply by inputting a training script. This method is simple, fast, and efficient for users, and it does not rely on a benchmark accelerator card, reducing labor and hardware costs.

[0189] Furthermore, users can customize the operator detection system 1000 if needed. During the configuration phase, when the S320 sends the training script, it can also send configuration information to request the completion of the custom configuration. The configuration information can be configured through a visual interface, implemented by the user-written configuration file, or implemented in other ways; this application does not impose specific limitations.

[0190] Optionally, the configuration information includes the storage path for the test report and comparison results. It may also include options for whether to detect the forward or backward computation process. It may also include the processor type for comparison, such as NPU, CPU, GPU, etc. It may also include the export level of the test report. It may also include an option for whether to replace the first output result with the second output result obtained from the up-precision calculation. It may also include the running time interval of the operator detection system 1000. It may also include whether to run the operator detection system 1000 for operator detection under specified training steps. It may also include the selection of the perturbation algorithm. It may also include whether to enable the threshold adjustment algorithm. It may also include the M value for each round of threshold adjustment. It may also include the maximum number of rounds of threshold adjustment, etc. The above configuration information is for illustrative purposes only and is not specifically limited in this application.

[0191] As mentioned above, the operators to be detected include floating-point operators and hotspot operators. For floating-point operators, the operator detection method provided in this application can quickly identify operators that may have accuracy issues in the early stages of model development, and promptly detect errors. For hotspot operators, the operator detection method provided in this application can continuously evaluate the accuracy of some key operators that may have problems during the long-term development and maintenance of model training, especially in generation environments or high-load training tasks, ensuring the stability and reliability of the entire training process. Even during large model cluster training, operator accuracy can be monitored and diagnosed in a normalized and stable manner.

[0192] In summary, the operator detection method provided in this application monitors the running progress of the training script on the accelerator card through an operator detection device. When the program reaches the operator to be detected, it obtains the first input data and the first output result of the operator to be detected. Then, it performs a slight perturbation on the first input data to obtain the second input data, and inputs the second input data into the operator to be detected. The operator to be detected is then run again to obtain the second output result. By comparing the first output result before the perturbation and the second output result after the perturbation, the operator detection result of the accelerator card is obtained. This makes the operator detection of the accelerator card no longer dependent on the benchmark accelerator card, and the operator detection method is no longer limited. It can cover all possible detection scenarios and meet the user's detection needs.

[0193] The operator detection method provided in this application has been described in detail above. The following section will combine... Figure 6 The operator detection device provided in this application will be described. This operator detection device can be... Figures 1-5 The operator detection device 120 described in the text.

[0194] Figure 6 This is a schematic diagram of the structure of an operator detection device provided in this application, as shown below. Figure 6As shown, the operator detection device 120 includes a receiving unit 121, an acquisition unit 122, a disturbance unit 123, a running unit 124, a comparison unit 125, and a threshold adjustment unit 126. It should be understood that... Figure 6 This is an exemplary division method. The operator detection device 120 can be divided into more or fewer unit modules, and this application does not make any specific limitations.

[0195] The receiving unit 121, acquiring unit 122, perturbation unit 123, running unit 124, comparison unit 125, and threshold adjustment unit 126 can all be implemented in software or in hardware. For example, the implementation of threshold adjustment unit 126 will be described below. Similarly, the implementation of receiving unit 121, acquiring unit 122, perturbation unit 123, running unit 124, and comparison unit 125 can refer to the implementation of threshold adjustment unit 126.

[0196] The threshold adjustment unit 126, as an example of a software functional unit, may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, or a container. Further, the aforementioned computing instance may be one or more. For example, the threshold adjustment unit 126 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0197] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0198] The threshold adjustment unit 126 is an example of a hardware functional unit. The threshold adjustment unit 126 may include at least one computing device, such as a server. Alternatively, the threshold adjustment unit 126 may be implemented using a central processing unit (CPU), an application-specific integrated circuit (ASIC), or a programmable logic device (PLD). The aforementioned PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), a data processing unit (DPU), a neural network processing unit (NPU), a system-on-a-chip (SoC), an offload card, an accelerator card, or any combination thereof.

[0199] The threshold adjustment unit 126 includes multiple computing devices that can be distributed in the same region or in different regions. Similarly, the threshold adjustment unit 126 can be distributed in the same Availability Zone (AZ) or in different AZs. Likewise, the threshold adjustment unit 126 can be distributed in the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, GALs, DPUs, NPUs, SoCs, offloading cards, and accelerator cards.

[0200] It should be noted that, in other embodiments, the steps implemented by the receiving unit 121, the acquiring unit 122, the perturbation unit 123, the running unit 124, the comparison unit 125, and the threshold adjustment unit 126 can be specified as needed. The receiving unit 121, the acquiring unit 122, the perturbation unit 123, the running unit 124, the comparison unit 125, and the threshold adjustment unit 126 respectively implement different steps in the operator detection method to realize all the functions of the operator detection device.

[0201] The functions of the receiving unit 121, the acquisition unit 122, the disturbance unit 123, the running unit 124, the comparison unit 125, and the threshold adjustment unit 126 are explained below.

[0202] The receiving unit 121 is used to receive calculation instructions sent by the processor, and to run multiple operators based on the calculation instructions. The calculation instructions are generated by the processor according to the training script of the artificial intelligence (AI) model, and are specifically used to implement... Figure 5 S510 to S520 and their optional steps in the embodiments.

[0203] The acquisition unit 122 is used to acquire the first input data and the first output result of the operator to be detected when the operation reaches the operator to be detected. Specifically, it is used to implement... Figure 5 S530 and its optional steps in the embodiments.

[0204] The perturbation unit 123 is used to perturb the first input data to obtain the second input data, specifically for implementing... Figure 5 S540 and its optional steps in the embodiments.

[0205] The execution unit 124 is used to input the second input data into the first operator, run the first operator to obtain the second output result, and is specifically used to implement... Figure 5 S550 and its optional steps in the embodiments.

[0206] The comparison unit 125 is used to compare the first output result with the second output result to obtain the operator detection result, specifically for implementing... Figure 5 S560 to S590 in the embodiment Figure 4 Examples and optional steps thereof.

[0207] In one possible implementation, the acquisition unit 122 is used to monitor the running progress of multiple operators in the operator list. When the operation reaches the operator to be detected, the first input data and the first output result of the operator to be detected are acquired. The operator list includes multiple floating-point operators and multiple hot-spot operators. The floating-point operators include operators used to process floating-point operations. The hot-spot operators include operators with historical precision problems and operators expected to have precision problems in the future.

[0208] In one possible implementation, the comparison unit 125 is used to obtain the comparison threshold of the operator to be detected based on the threshold list, compare the difference between the first output result and the second output result with the comparison threshold, and obtain the operator detection result. The operator detection result includes information about abnormal operators. If the difference value is greater than the threshold, the operator to be detected is determined to be an abnormal operator. The threshold list includes the comparison threshold corresponding to the operator. The comparison threshold is obtained by the accelerator card after attenuating the initial threshold through a multi-round threshold adjustment method before receiving the calculation instructions sent by the processor.

[0209] In one possible implementation, the threshold adjustment unit 126 is used to run the operator to be detected to obtain a first output result before perturbation and a second output result after perturbation, determine the difference value between the first output result and the second output result, and determine the current round threshold as the comparison threshold of the operator to be detected if the comparison result between the difference value and the current round threshold reaches the stopping condition. The current round threshold includes the initial threshold or the threshold obtained from the previous round threshold adjustment. If the comparison result between the difference value and the current round threshold does not reach the stopping condition, the current round threshold is attenuated to obtain the next round threshold. Specifically, it is used to implement... Figure 3 S340 in the embodiment Figure 4 Examples and optional steps thereof.

[0210] In one possible implementation, the threshold adjustment unit 126 is used to acquire the third output result corresponding to the first input data obtained by the processor running the operator to be detected. When the first output result and the third output result are consistent, it determines the difference between the first output result and the second output result. When the first output result and the third output result are inconsistent, it uses the minimum value between the difference value and the threshold of this round as the comparison threshold. Specifically, it is used to implement... Figure 3 S340 in the embodiment Figure 4 Examples and optional steps thereof.

[0211] In one possible implementation, the threshold adjustment unit 126 is used to run the operator to be detected multiple times, obtain M first output results and M second output results, obtain M difference values, and obtain M third output results obtained by the processor running the operator to be detected multiple times. When the M third output results are inconsistent with the M first output results, the threshold adjustment unit determines the comparison threshold of the operator to be detected based on the M difference values; when all M third output results are consistent with the M first output results, the threshold for the next round is determined based on the M difference values ​​and the current round threshold. Specifically, it is used to implement... Figure 3 S340 in the embodiment Figure 4 Examples and optional steps thereof.

[0212] In one possible implementation, the threshold adjustment unit 126 is used to, when all M third output results are inconsistent with the M first output results, use the minimum value among the M difference values ​​as the comparison threshold. The threshold adjustment unit 126 is also used to, when there are partial inconsistencies between the M third output results and the M first output results, divide the M difference values ​​into a consistent group and an inconsistent group, wherein at least one difference value in the consistent group corresponds to a third output result that is consistent with the first output result, and at least one difference value in the inconsistent group corresponds to a third output result that is inconsistent with the first output result. The threshold adjustment unit 126 is used to determine the comparison threshold based on the maximum value of at least one difference value in the consistent group and the minimum value among at least one difference value in the inconsistent group.

[0213] In one possible implementation, the receiving unit 121 is used to receive a detection instruction sent by the processor, and the acquiring unit 122 is used to acquire the first input data and the first output result of the operator to be detected based on the detection instruction. The calculation instruction and the detection instruction are generated by the processor based on a machine learning framework, which includes operator detection device code. Specifically, this is used for implementation... Figure 3 S310 in the embodiment Figure 5 S510, S520 and their optional steps in the embodiments.

[0214] The operator detection device 120 may further include a threshold list 127 and an operator list 128. The operator list 128 includes multiple operators to be detected, and the threshold list 127 includes a comparison threshold corresponding to each operator to be detected. The operator list is used by the acquisition unit 122 to monitor the operator's running progress and acquire the first input data and first output result of the operator to be detected. The threshold list 127 is used by the comparison unit 125 to determine whether the operator is abnormal, thereby generating a detection report. The threshold list 127 can be determined by the threshold adjustment unit 126, as detailed in the reference. Figure 4 The description of the embodiments is not specifically limited in this application.

[0215] In summary, the operator detection device provided in this application monitors the running progress of the training script on the accelerator card. When the program reaches the operator to be detected, it obtains the first input data and the first output result of the operator to be detected. Then, it performs a slight perturbation on the first input data to obtain the second input data. The second input data is then input into the operator to be detected, and the operator to be detected is run again to obtain the second output result. By comparing the first output result before the perturbation and the second output result after the perturbation, the operator detection result of the accelerator card is obtained. This makes the operator detection of the accelerator card no longer dependent on the benchmark accelerator card, and the operator detection method is no longer limited. It can cover all possible detection scenarios and meet the user's detection needs.

[0216] The operator detection system, method, and apparatus provided in this application have been described in detail above. The following section, in conjunction with... Figures 7-9 The computing device provided in this application will be explained.

[0217] Figure 7 This is a schematic diagram of the structure of a computing device provided in this application, such as... Figure 7 As shown, the computing device 700 includes a bus 702, a processor 704, a memory 706, and a communication interface 708. The processor 704, memory 706, and communication interface 708 communicate with each other via the bus 702. The computing device 700 can be a server. It should be understood that this application does not limit the number of processors, memory, and accelerator cards in the computing device 700. The computing device 700 can be used to deploy the aforementioned... Figures 1-6 Operator detection device 120 in the embodiment.

[0218] The 702 bus can be a Peripheral Component Interconnect Express (PCIe) bus, an Extended Industry Standard Architecture (EISA) bus, a Unified Bus (Ubus or UB), a Compute Express Link (CXL) bus, a Cache Coherent Interconnect for Accelerators (CCIX) bus, etc. The Unified Bus is also known as the Lingqu bus. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 7 The bus 702 is represented by only one line, but this does not mean that there is only one bus or one type of bus. The bus 702 may include a path for transmitting information between various components of the computing device 700 (e.g., memory 706, processor 704, communication interface 708). The unified bus may also be called the Lingqu bus.

[0219] The processor 704 may include any one or more computing devices such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP) or a digital signal processor (DSP), an ASIC, an FPGA, a CPLD, an NPU, a SoC, an offload card, or an accelerator card.

[0220] Memory 706 may include volatile memory, such as random access memory (RAM). Memory 706 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD). Furthermore, memory 706 may also be implemented using storage class memory (SCM), phase change memory (PCM), or other types of storage media.

[0221] It is worth noting that the same type of storage medium can be configured in the same computing device to realize the function of memory 706, or two or more types of storage media can be configured to realize the function of memory 706. This application does not limit this.

[0222] Memory 706 is used to store instructions, and processor 704 is used to execute the instructions stored in memory 706. Memory 706 stores program code, and processor 704 can call the program code stored in memory 706 to implement... Figures 1-6 The operation steps performed by the processor 200 in this embodiment. For example... Figure 3 S310 to S360 in the embodiment Figure 5 S510, S520, S580, and S590 and their optional steps in the embodiments.

[0223] The communication interface 708 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the computing device 700 and other devices or communication networks.

[0224] The 707 accelerator card can be implemented using computing devices such as DPUs, NPUs, GPUs, XPUs, and SoCs, or computing units such as AI chips. The 707 accelerator card includes a processing unit and a power supply circuit. The power supply circuit powers the processing unit, enabling it to perform its processing tasks. Figures 1-6 The operation steps performed by the accelerator card 100 in this embodiment. Specifically, the accelerator card 707 can achieve... Figure 6 The embodiments describe the functions of the receiving unit, acquiring unit, perturbation unit, running unit, comparison unit, and threshold adjustment unit. The receiving unit receives computation instructions sent by the processor and runs multiple operators based on these instructions. These computation instructions are generated by the processor according to the training script of the artificial intelligence (AI) model and are specifically used to implement… Figure 5S510, S520 and their optional steps in the embodiment; the acquisition unit is used to acquire the first input data and the first output result of the operator to be detected when running to the operator to be detected, specifically used to implement Figure 5 S530 of the embodiment and its optional steps; the perturbation unit is used to perturb the first input data to obtain the second input data, specifically for implementing Figure 5 S540 and its optional steps in the embodiment; the running unit is used to input the second input data into the first operator, run the first operator to obtain the second output result, specifically used to implement Figure 5 S550 of the embodiment and its optional steps; the comparison unit is used to compare the first output result with the second output result to obtain the operator detection result, specifically used to implement Figure 5 The steps S560, S570 and their optional steps in the embodiment will not be described in detail here.

[0225] In this embodiment, the code of the above-mentioned unit module can be stored in the memory 706, and the processor 704 can execute code generation instructions and send them to the accelerator card 707. The accelerator card 707 executes the instructions to realize the function of the above-mentioned unit module.

[0226] This application also provides a cluster of computing devices. For example... Figure 8 As shown, Figure 8 This is an example diagram of a computing device cluster provided in this application, which includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0227] As one possible implementation, the same implementation can be stored in the memory 706 of one or more computing devices 700 in a computing device cluster. Figures 1 to 6 The instructions for the operation steps that the processor needs to implement in the described operator detection method. Alternatively, different implementations may be stored in the memory 706 of one or more computing devices 700. Figures 1 to 6 The described operator detection method includes instructions for the operation steps that the processor needs to implement. In other words, a combination of one or more computing devices 700 can jointly execute instructions for implementing the operation steps that the processor needs to implement in the operator detection method.

[0228] As one possible implementation, the same implementation can be stored in the accelerator cards 707 of one or more computing devices 700 in the computing device cluster. Figures 1 to 6The described operator detection method specifies the instructions for the operation steps that the accelerator card needs to implement. Alternatively, one or more accelerator cards 707 of the computing device 700 may each store different implementations. Figures 1 to 6 The described operator detection method includes instructions for the operation steps that the accelerator card needs to implement. In other words, a combination of one or more computing devices 700 can jointly execute instructions for implementing the operation steps that the accelerator card needs to implement in the operator detection method.

[0229] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN), a local area network (LAN), or similar. Figure 9 One possible implementation is shown. For example... Figure 9 As shown, Figure 9 This is a schematic diagram of another computing device cluster structure provided in this application. Two computing devices, 700A and 700B, are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device.

[0230] In this type of possible implementation, the accelerator card 707 in computing device 700B is used to implement the functions of the receiving unit, the acquisition unit, the perturbation unit, and the operation unit. Meanwhile, the accelerator card 707 in computing device 700A is used to implement the functions of the receiving unit, the acquisition unit, the perturbation unit, the operation unit, the comparison unit, and the threshold adjustment unit.

[0231] Figure 9 The connection method between the computing device clusters shown can be implemented by sending the comparison results of computing device 700B to computing device 700A, taking into account that model training is usually distributed training, in order to reduce the processing pressure of each device.

[0232] It should be understood that Figure 9 The functions of the computing device 700A shown can also be performed by multiple computing devices 700. Similarly, the functions of the computing device 700B can also be performed by multiple computing devices 700.

[0233] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to execute a reasoning method based on a large language model.

[0234] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute a reasoning method based on a large language model, or instruct the computing device to execute a reasoning method.

[0235] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. An operator detection method, characterized by, The method comprises: The acceleration card receives a calculation instruction sent by a processor, runs a plurality of operators based on the calculation instruction, and the calculation instruction is generated by the processor according to a training script of an artificial intelligence AI model; In the case that the acceleration card runs to a to-be-detected operator, the acceleration card acquires first input data and first output results of the to-be-detected operator; The acceleration card obtains second input data by perturbing the first input data; The acceleration card inputs the second input data into the first operator, runs the first operator to obtain second output results; The acceleration card compares the first output results with the second output results to obtain an operator detection result.

2. The method of claim 1, wherein, In the case that the acceleration card runs to a to-be-detected operator, the acceleration card acquires first input data and first output results of the to-be-detected operator comprises: The acceleration card monitors the running progress of the plurality of operators based on an operator list, and acquires the first input data and the first output results of the to-be-detected operator when running to the to-be-detected operator, wherein the operator list comprises a plurality of floating point operators and a plurality of hot point operators, the floating point operators comprise operators for processing floating point operations, and the hot point operators comprise operators that have historically existed abnormities and operators that are predicted to have abnormal problems in the future.

3. The method according to claim 1 or 2, characterized in that, The acceleration card compares the first output results with the second output results to obtain an operator detection result comprises: The acceleration card acquires a comparison threshold value of the to-be-detected operator based on a threshold value list, compares a difference value between the first output results and the second output results with the comparison threshold value to obtain the operator detection result, wherein the operator detection result comprises information of an abnormal operator, the to-be-detected operator is determined as an abnormal operator in the case that the difference value is greater than the threshold value, the threshold value list comprises a comparison threshold value corresponding to an operator, and the comparison threshold value is obtained by attenuating an initial threshold value through a plurality of rounds of threshold value adjustment methods before the acceleration card receives the calculation instruction sent by the processor.

4. The method of claim 3, wherein, Each round of threshold value adjustment method in the plurality of rounds of threshold value adjustment methods comprises: The acceleration card runs the to-be-detected operator to obtain the first output results before perturbation and the second output results after perturbation; The acceleration card determines a difference value between the first output results and the second output results; In the case that a comparison result between the difference value and a current round threshold value reaches a stop condition, the acceleration card determines the current round threshold value as the comparison threshold value of the to-be-detected operator, wherein the current round threshold value comprises the initial threshold value or a threshold value obtained through a last round of threshold value adjustment; In the case that the comparison result between the difference value and the current round threshold value does not reach the stop condition, the acceleration card attenuates the current round threshold value to obtain a next round threshold value.

5. The method of claim 4, wherein, The acceleration card determines a difference value between the first output results and the second output results comprises: The acceleration card acquires third output results corresponding to the first input data obtained by the processor running the to-be-detected operator; In a case where the first output result is consistent with the third output result, the accelerator card determines a difference value between the first output result and the second output result; The method further comprises: In a case where the first output result is inconsistent with the third output result, taking the minimum value of the difference value and the threshold value of the current round as a comparison threshold.

6. The method according to claim 4 or 5, characterized in that, Each of the plurality of threshold value adjustment methods comprises: The accelerator card runs the to-be-detected operator multiple times to obtain M difference values between M first output results and M second output results; The accelerator card obtains M third output results obtained by the processor running the to-be-detected operator multiple times; In a case where the M third output results are inconsistent with the M first output results, the accelerator card determines a comparison threshold value of the to-be-detected operator based on the M difference values; In a case where the M third output results are all consistent with the M first output results, the accelerator card determines a threshold value of a next round based on the M difference values and the threshold value of the current round.

7. The method of claim 6, wherein, The accelerator card determines a comparison threshold value of the to-be-detected operator based on the M difference values in a case where the M third output results are inconsistent with the M first output results, comprising: In a case where the M third output results are all inconsistent with the M first output results, the accelerator card takes the minimum value of the M difference values as the comparison threshold value; In a case where the M third output results are partially inconsistent with the M first output results, the accelerator card divides the M difference values into a consistent group and an inconsistent group, wherein at least one difference value in the consistent group corresponds to a third output result and a first output result that are consistent, and at least one difference value in the inconsistent group corresponds to a third output result and a first output result that are inconsistent; The accelerator card determines the comparison threshold value based on the maximum value of at least one difference value in the consistent group and the minimum value of at least one difference value in the inconsistent group.

8. The method according to any one of claims 1 to 7, characterized in that, In a case where the accelerator card runs to a to-be-detected operator, the accelerator card obtains first input data and a first output result of the to-be-detected operator, comprising: The accelerator card receives a detection instruction sent by the processor, and obtains first input data and a first output result of the to-be-detected operator based on the detection instruction, wherein the calculation instruction and the detection instruction are generated by the processor based on a machine learning framework, and the machine learning framework comprises operator detection device code.

9. An operator detection device, characterized in that The device comprises: A receiving unit configured to receive a calculation instruction sent by a processor, and run a plurality of operators based on the calculation instruction, wherein the calculation instruction is generated by the processor according to a training script of an artificial intelligence (AI) model; An obtaining unit configured to, in a case where the accelerator card runs to a to-be-detected operator, obtain first input data and a first output result of the to-be-detected operator; A perturbation unit configured to perturb the first input data to obtain second input data; A running unit configured to input the second input data to the first operator, and run the first operator to obtain a second output result; The comparison unit is configured to compare the first output result with the second output result to obtain an operator detection result.

10. The apparatus of claim 9, wherein, The acquisition unit is configured to monitor running progress of the plurality of operators in an operator list, and acquire first input data and a first output result of the to-be-detected operator when the to-be-detected operator is running.

11. The apparatus of claim 9 or 10, wherein, The comparison unit is configured to acquire a comparison threshold of the to-be-detected operator based on a threshold list, compare a difference value between the first output result and the second output result with the comparison threshold, and obtain the operator detection result, wherein the operator detection result includes information of an abnormal operator, the to-be-detected operator is determined as the abnormal operator when the difference value is greater than the threshold, and the threshold list includes a comparison threshold corresponding to an operator, and the comparison threshold is obtained by attenuating an initial threshold through a plurality of rounds of threshold adjustment methods before the accelerator receives a calculation instruction sent by the processor.

12. The apparatus of claim 11, wherein, The device further includes a threshold adjustment unit, The threshold adjustment unit is configured to run the to-be-detected operator to obtain the first output result before perturbation and the second output result after perturbation. The threshold adjustment unit is configured to determine a difference value between the first output result and the second output result. The threshold adjustment unit is configured to determine the current round threshold as the comparison threshold of the to-be-detected operator when a comparison result between the difference value and the current round threshold reaches a stop condition, wherein the current round threshold includes the initial threshold or a threshold obtained through last round of threshold adjustment. The threshold adjustment unit is configured to attenuate the current round threshold to obtain the next round threshold when the comparison result between the difference value and the current round threshold does not reach the stop condition.

13. The device of claim 12, wherein The threshold adjustment unit is configured to acquire a third output result corresponding to the first input data obtained by running the to-be-detected operator by the processor. The threshold adjustment unit is configured to determine a difference value between the first output result and the second output result when the first output result is consistent with the third output result. The threshold adjustment unit is configured to take a minimum value between the difference value and the current round threshold as a comparison threshold when the first output result is inconsistent with the third output result.

14. The device of claim 12 or 13, wherein The threshold adjustment unit is configured to run the to-be-detected operator multiple times to obtain M difference values between M first output results and M second output results. The threshold adjustment unit is configured to acquire M third output results obtained by running the to-be-detected operator multiple times by the processor. The threshold adjustment unit is configured to determine the comparison threshold of the to-be-detected operator based on the M difference values in a case where the M third output results are inconsistent with the M first output results. The threshold adjustment unit is configured to determine a next round threshold based on the M difference values and the current round threshold in a case where the M third output results are all consistent with the M first output results.

15. The method of claim 14, wherein, The threshold adjustment unit is configured to take the minimum value of the M difference values as the comparison threshold in a case where the M third output results are all inconsistent with the M first output results. The threshold adjustment unit is configured to divide the M difference values into a consistent group and an inconsistent group in a case where the M third output results are partially inconsistent with the M first output results, wherein the third output result corresponding to at least one difference value in the consistent group is consistent with the first output result, and the third output result corresponding to at least one difference value in the inconsistent group is inconsistent with the first output result. The threshold adjustment unit is configured to determine the comparison threshold based on the maximum value of at least one difference value in the consistent group and the minimum value of at least one difference value in the inconsistent group.

16. The apparatus of any one of claims 9-15, wherein, The receiving unit is configured to receive a detection instruction sent by the processor. The obtaining unit is configured to obtain first input data and a first output result of the to-be-detected operator based on the detection instruction, wherein the calculation instruction and the detection instruction are generated by the processor based on a machine learning framework, and the machine learning framework includes operator detection apparatus code.

17. A chip, characterized by The chip includes a power supply circuit and a processing unit, the power supply circuit is used for powering the processing unit, and the processing unit is used for executing the operation steps of the method of any one of claims 1-8.

18. A computing device, comprising: The computing device includes a processor and an acceleration card, and the acceleration card executes the operation steps of the method of any one of claims 1-8.

19. A computer-readable storage medium, characterized in that, The computer program instructions, when executed by a computing device or a cluster of computing devices, cause the computing device or the cluster of computing devices to execute the operation steps of the method of any one of claims 1-8.

20. A computer program product comprising instructions, characterized in that, The instructions, when executed by a computing device or a cluster of computing devices, cause the computing device or the cluster of computing devices to execute the operation steps of the method of any one of claims 1-8.

Images