Neural network operator testing, live application method and apparatus, device, medium

Abstract

The application discloses a neural network operator test, live application method and device, equipment and medium. The test method comprises the following steps: obtaining a type identifier of a corresponding neural network operator in response to an operator test instruction; querying an operator instance template corresponding to the type identifier from an operator template library, wherein the operator instance template comprises instance definition data corresponding to a plurality of operator instances of the neural network operator; running a standard test case corresponding to the neural network operator; testing each operator instance constructed according to the instance definition data respectively; and obtaining execution index data corresponding to each operator instance. The application selects the operator instance template and the standard test case according to the type to test the neural network operator, so that the test efficiency is higher, the test and deployment cost of the deep learning model can be reduced, the stability and correctness of the neural network operator are maximized, and the deep learning model required by the network live scene can be constructed.

Description

Technical Field

[0001] This application relates to the field of live streaming technology, and in particular to a neural network operator testing, live streaming application method, and corresponding apparatus, electronic equipment, and computer-readable storage medium. Background Technology

[0002] In online live streaming scenarios, streamers push video streams to the live streaming room to achieve application purposes such as talent display, information sharing, and knowledge education. This allows streamers to participate in social labor and earn income through these activities, thus promoting overall social benefits.

[0003] The various technologies used in live streaming, including natural language processing, image processing, and speech processing, widely employ various deep learning models based on neural networks. These deep learning models are deployed through deep learning inference engines. Deep learning inference engines enable computer devices to learn to use models to achieve corresponding functions. They are systems composed of several operator units. Through the engine, deep learning models are parsed into specific operator units, enabling deep learning models to be deployed and executed on hardware.

[0004] Before deploying the model, it is necessary to test each neural network operator. Under the existing testing method, for the same type of neural network operator but with different backend configuration conditions, different structural definitions of video network operators are required, and different test cases need to be written accordingly, which makes the testing efficiency very low.

[0005] In view of this, it is necessary to improve the testing methods for neural network operators in related fields in order to improve testing efficiency and adapt to the application needs of rapidly developing deep learning models in scenarios such as live streaming. Summary of the Invention

[0006] The primary objective of this application is to address at least one of the aforementioned problems by providing a method for testing neural network operators, live streaming applications, and corresponding apparatus, electronic devices, and computer-readable storage media.

[0007] To achieve the various objectives of this application, the following technical solution is adopted:

[0008] A method for testing neural network operators proposed to meet one of the purposes of this application includes the following steps:

[0009] Responding to the operator test command, obtain the type identifier of the corresponding neural network operator;

[0010] Retrieve the operator instance template corresponding to the type identifier from the operator template library. The operator instance template contains instance definition data corresponding to multiple operator instances of the neural network operator.

[0011] Run the standard test cases corresponding to the neural network operators to test each operator instance constructed based on the instance definition data, and obtain the execution index data corresponding to each operator instance.

[0012] Optionally, before responding to the operator test instruction and obtaining the type identifier of the corresponding neural network operator, the following steps are included:

[0013] The code information of the neural network operator is detected to determine the various structural definition attributes of the neural network operator;

[0014] Obtain the standard definition data provided for each structural definition attribute, and construct the instance definition data of the neural network operator so that it contains the standard definition data corresponding to each result definition attribute;

[0015] Using the instance definition data as a template, other instance definition data are derived by modifying some of the standard definition data with random numbers;

[0016] All instance definition data are constructed into operator instance templates of the neural network operator, and stored in the operator template library by mapping them to the type identifier of the neural network operator.

[0017] Optionally, after obtaining the execution metric data corresponding to each operator instance, the following are included:

[0018] The execution metric data corresponding to each operator instance is compared with the corresponding preset real metric data, and the test difference between the execution metric data and the real metric data of each operator instance is calculated.

[0019] Select multiple operator instances whose test difference is less than a preset threshold as qualified operator instances, and obtain the instance definition data corresponding to the qualified operator instances;

[0020] Configure the neural network operator with the instance definition data of the qualified operator instance, and use it for the deployment of the neural network model.

[0021] Optionally, after obtaining the execution metric data corresponding to each operator instance, the following are included:

[0022] The execution metric data corresponding to each operator instance is compared with the corresponding preset real metric data, and the test difference between the execution metric data and the real metric data of each operator instance is calculated.

[0023] Operator instances whose test differences are greater than or equal to a preset threshold are identified as abnormal operator instances;

[0024] Obtain instance definition data corresponding to the abnormal operator instance from the operator instance template, and adjust the instance definition data to derive multiple derived instance definition data;

[0025] The standard test cases are invoked to test the operator instances constructed based on the derived instance definition data, and the corresponding execution indicator data is obtained.

[0026] Based on the execution index data, a qualified operator instance is found among the operator instances derived from each abnormal operator instance, and a mapping relationship data is constructed between the qualified operator instance and the abnormal operator instance.

[0027] Optionally, after obtaining the execution metric data corresponding to each operator instance, the following are included:

[0028] The execution metric data is obtained by testing each operator instance defined by the operator instance template of the same type of neural network operator under different backend configuration conditions. The backend configuration conditions are distinguished by at least one of the following: backend type identifier, operation precision level, and data arrangement type. The execution metric data includes the execution time metric of the corresponding operator instance.

[0029] Based on the execution time metric in the execution metric data, the average execution time corresponding to each backend configuration condition is calculated.

[0030] Generate and display a comparison chart showing the average execution time for different backend configuration conditions.

[0031] Optionally, before running the standard test cases corresponding to the neural network operator, the following steps are included:

[0032] Determine the backend type identifier in the backend configuration conditions from the name of the neural network operator;

[0033] Query the registration configuration table to determine if the backend type identifier has been pre-registered. If not, terminate the execution; otherwise, continue the execution.

[0034] A method for live streaming applications of neural network operators, proposed to meet one of the purposes of this application, includes the following steps:

[0035] An image encoder is used to represent the image feature information of image frames in a live video stream of a network broadcast. The image encoder contains the neural network operator obtained by the neural network operator testing method.

[0036] The image feature information is used to perform classification mapping to determine the category information corresponding to the image frame.

[0037] A neural network operator testing apparatus provided for one of the purposes of this application includes:

[0038] The instruction response module is configured to respond to operator test instructions and obtain the type identifier of the corresponding neural network operator.

[0039] The template query module is configured to query the operator instance template corresponding to the type identifier from the operator template library. The operator instance template contains instance definition data corresponding to multiple operator instances of the neural network operator.

[0040] The test execution module is configured to run the standard test cases corresponding to the neural network operators, and to test each operator instance constructed based on the instance definition data to obtain the execution index data corresponding to each operator instance.

[0041] An electronic device provided for one of the purposes of this application includes a central processing unit and a memory, the central processing unit being configured to invoke and run a computer program stored in the memory to perform the steps of the neural network operator testing method described in this application.

[0042] A computer-readable storage medium is provided for another purpose of this application, which stores, in the form of computer-readable instructions, a computer program implemented according to the neural network operator testing method, which, when invoked by a computer, performs the steps included in the method.

[0043] A computer program product provided for another purpose of this application includes a computer program / instructions that, when executed by a processor, implement the steps of the method described in any embodiment of this application.

[0044] Compared to existing technologies, this application sets corresponding operator instance templates for neural network operators in deep learning models according to their categories. Each operator instance template provides instance definition data describing various operator instances corresponding to that type of neural network operator. When testing a neural network operator, the corresponding operator instance template is retrieved from the operator template library based on the operator's type identifier. Standard test cases corresponding to the type identifier are then applied to perform unified testing on all operator instances defined by the instance definition data in the operator instance template. This eliminates the need to apply test cases to individual instances, resulting in higher testing efficiency, reduced testing and deployment costs for deep learning models, and avoidance of multi-source test case editing. This maximizes the stability and correctness of neural network operators, achieving significant economic benefits for applications such as live streaming that require extensive use of deep learning models. Attached Figure Description

[0045] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0046] Figure 1 This is an exemplary network architecture used for the live streaming service in the live streaming scenario of this application;

[0047] Figure 2 This is a flowchart illustrating one embodiment of the neural network operator testing method of this application.

[0048] Figure 3 This is a schematic diagram illustrating the process of constructing the operator template library in an embodiment of this application;

[0049] Figure 4 This is a flowchart illustrating the process of identifying qualified operator instances in an embodiment of this application;

[0050] Figure 5 This is a flowchart illustrating the process of implementing extended testing to recommend qualified operator instances for anomalous operator instances in an embodiment of this application;

[0051] Figure 6 This is a flowchart illustrating the execution metric data under different backend configuration conditions in the embodiments of this application;

[0052] Figure 7 This is a schematic block diagram of the neural network operator testing device of this application;

[0053] Figure 8 This is a schematic diagram of the structure of an electronic device used in this application. Detailed Implementation

[0054] Please see Figure 1 The network architecture adopted in an exemplary application scenario of this application includes a terminal device 80, a media server 81, and an application server 82. The terminal device 80 can be used to run a live streaming terminal program, allowing broadcasters or viewers to use live streaming functions. For example, a broadcaster can upload a live video stream to the media server 81 through their terminal device 80, or the media server 81 can push a target user's live video stream to a viewer's terminal device 80 for playback. The media server 81 is mainly responsible for pushing the live video streams of each broadcaster to their respective live streaming rooms. The application server 82 can be used to deploy network live streaming services to maintain interaction between broadcasters and viewers based on the live streaming room.

[0055] Image frames in the live video stream can be feature-represented using a deep learning model based on neural networks to obtain their corresponding image feature information. This information can then be adapted to the needs of different downstream tasks, enabling various applications such as face detection, violation identification, content recognition, and type recognition, thus providing corresponding data for various applications. Each application's corresponding deep learning model uses a classifier to categorize the image feature information. For example, classification determines the violation label corresponding to the image feature information, whether there is a face image, etc. The deep learning model is built from various neural network operators, including but not limited to convolution, pooling, activation, and fully connected operations.

[0056] Different neural network operators have their corresponding data processing capabilities. These operators can be implemented through program code and combined according to a certain structure to form corresponding deep learning models. Then, they are encoded by relevant compilers and run on hardware devices. A neural network operator can be viewed as a mathematical computation function. Given one or a set of input data, it performs calculations according to certain rules to obtain the corresponding output data.

[0057] Each neural network operator can operate under different backend configurations. These different backend configurations can be distinguished by at least one of the following: backend type identifier, computational precision level, and data arrangement type. For example, for a neural network operator performing convolution operations, the backend type identifier can be arm v7, arm v8, metal, opengl, SSE, etc., corresponding to different chip hardware types; the computational precision level can be float32, float16, int16, int8, etc., corresponding to different computational precisions; and the data arrangement type can be nchw, nc4hw4, nc8hw8, etc., corresponding to common data arrangement methods in the industry. It is evident that neural network operators exhibit diversity in adapting to different backend configurations.

[0058] Each neural network operator can also demonstrate diversity in adapting to its own structural changes. Taking the neural network operator corresponding to the convolution operation as an example, the width, height, and number of channels of its input feature map, as well as the number of channels of its output feature map, the width, height, and sliding stride of the convolution kernel can all be varied. Each variation corresponds to the definition of an operator instance, thus allowing each neural network operator to have a large number of different operator instances.

[0059] The computer program product implemented according to the neural network operator testing method of this application can run on any electronic device with sufficient computing power. By running the computer program product, the various steps of the method are executed to carry out efficient testing of the neural network operator, thereby shortening the production cycle of the corresponding deep learning model, enabling the rapid deployment of intelligent recognition technology for live video streams, serving application scenarios such as live streaming, and rapidly improving the user experience.

[0060] Based on the above exemplary scenarios and related principle descriptions, please refer to Figure 2 In one embodiment of the neural network operator testing method of this application, the following steps are included:

[0061] Step S1100: Respond to the operator test command and obtain the type identifier of the corresponding neural network operator;

[0062] When a developer needs to test a specific neural network operator, they can trigger an operator test command for that operator in the testing system. This operator test command includes a type identifier for the neural network operator. The type identifier indicates the operation type to which the neural network operator belongs, such as convolution, pooling, activation, or fully connected operations.

[0063] In one embodiment, neural network operators are named according to certain naming rules during the development phase. These names include a type identifier corresponding to the operation type and other relevant information, such as the backend configuration conditions applicable to the neural network operator. When a developer specifies a neural network operator and triggers the corresponding operator test command, the type identifier can be determined from the name of the neural network operator.

[0064] Developers can trigger the operator test instructions in various ways, such as through shortcut menus or operation controls provided by the graphical user interface, or by entering the corresponding executable instructions through the naming console. A single neural network operator's corresponding operator test instruction can be triggered individually, or multiple neural network operators' corresponding operator test instructions can be triggered simultaneously. For ease of understanding, this application mainly uses a single neural network operator's corresponding operator test instruction as an example, but the operating principle for triggering simultaneous testing of multiple neural network operators is the same as the testing process for a single neural network operator.

[0065] Step S1200: Query the operator instance template corresponding to the type identifier from the operator template library. The operator instance template contains instance definition data corresponding to multiple operator instances of the neural network operator.

[0066] As mentioned earlier, each neural network operator, due to the configurability of its free structural properties, can have its corresponding operator instance defined by configuring the data corresponding to its various structural definition attributes. For example, for a neural network operator corresponding to a convolution operation, its structural definition attributes can be represented by the width, height, and number of channels of its input feature map, the number of channels of its output feature map, and the width, height, and stride of the convolution kernel, as follows:

[0067] [W1,H1,C1,C2,Wk,Hk,Sk]

[0068] If the data of any structural definition attribute remains unchanged, changing the data of any structural definition attribute to another fixed value will also generate a new operator instance definition. Thus, by providing different data combinations for each structural definition attribute of the neural network operator to form the corresponding instance definition data, multiple operator instances of the neural network operator can be defined.

[0069] Based on the above method, a typical operator instance template can be defined as follows:

[0070] Args_Func_Conv2d_K1x1_S1=(

[0071] [16,16,1,1,1,16,32],

[0072] [16,16,1,1,1,32,32],

[0073] [16,16,1,1,1,32,64],

[0074] [16,16,1,1,1,64,64],

[0075] [32,32,1,1,1,16,16],

[0076] [32,32,1,1,1,16,32],

[0077] [32,32,1,1,1,32,32],)

[0078] In this context, the instance definition data of each operator instance is represented as a vector, and the correspondence between the value of each element in the vector and the structure definition attribute can be flexibly defined as needed.

[0079] The execution unit of the deep learning inference engine can construct corresponding operator instances for actual testing based on such instance definition data. Of course, the operator library of the deep learning inference engine pre-stores the code information and implementation of the neural network operators, so it can be called during testing to configure the corresponding operator instances as needed.

[0080] Accordingly, we can pre-abstract frequently used operator instances for each neural network operator, set the corresponding data for each structural definition attribute of the neural network operator to form corresponding instance definition data, and encapsulate these instance definition data into an operator instance template.

[0081] Following the above method, each neural network operator can be pre-defined with an operator instance template. An operator template library is created to store the operator instance templates of each neural network operator. The type identifier of each neural network operator and the operator instance template are constructed as mapping relationship data and stored in the operator instance template. Then, the type identifier can be used as an index item to query each operator instance template.

[0082] Since the operator test instruction carries the type identifier of the neural network operator to be tested, a query can be performed in the operator template library based on the type identifier to retrieve the operator instance template that maps to the type identifier.

[0083] It's easy to understand that since the operator instance template is created based on the overall operation type of the neural network operator, each neural network operator of each operation type uses the same operator instance template to define multiple operator instances in batches. There is no need to create specific operator instances of the neural network operator individually. The process of generating specific operator instances can be handled by the execution unit of the deep learning inference engine. This largely frees testers from the trouble of configuring operator instances themselves and can greatly improve testing efficiency.

[0084] Step S1300: Run the standard test cases corresponding to the neural network operator, and test each operator instance constructed according to the instance definition data to obtain the execution index data corresponding to each operator instance.

[0085] In addition to configuring a separate operator instance template for each neural network operator as a whole, this application also standardizes and abstracts the test code for various possible operator instances of the neural network operator, implementing them uniformly in the same standard test case. This ensures that each neural network operator has only one corresponding standard test case. By executing the standard test case, the test code of the standard test case identifies each operator instance of the neural network operator, and then calls the corresponding test business logic to perform the corresponding tests on the corresponding operator instances. Thus, each neural network operator is unified into a single standard test case.

[0086] When the standard test cases of the neural network operator are run, the instance definition data of each instance in the operator instance template of the neural network operator is passed to the execution unit in the deep learning inference engine. The execution unit is controlled to configure the specific implementation of the neural network operator using the instance definition data, thereby obtaining the corresponding operator instance. Then, the preset test data is input into the operator instance to run, obtain the corresponding result data and return it.

[0087] The test code of the standard test cases mainly transmits data to the execution unit through the RPC communication mechanism. Under the test framework of this application, the data is transmitted to the inference engine for calculation via RPC, and then the inference engine returns the result data to the test framework.

[0088] After obtaining the result data of the operator instance after execution from the execution unit, various performance parameters corresponding to the operator instance can be further analyzed, such as execution time. These result data and performance parameters can be regarded as generalized execution indicator data obtained by the corresponding operator instance. Of course, the specific content of the execution indicator data can be set as needed. Those skilled in the art can set the various result data and / or various performance parameters to be obtained according to the relevant aspects they need to examine, thereby defining the various execution indicator data they obtain.

[0089] In one embodiment, the testing framework can be responsible for comparing the result data obtained after running each operator instance with the pre-given real result data, and determining the corresponding execution index data based on the comparison results, such as determining various differences, etc. Similarly, those skilled in the art can flexibly set this according to actual needs.

[0090] It is easy to understand that, under the action of the standard test cases, each operator instance of the neural network operator is configured and tested accordingly, thereby obtaining the corresponding execution index data of each operator instance.

[0091] In one embodiment, the test framework of this application can be constructed based on the neural network operator testing method of this application, providing a main program entry point, and defining a registration layer, an execution layer, a shared parameter layer, and a common function layer. The functions of each part can be implemented according to the following principles:

[0092] The main program entry point serves as the primary entry point for the entire testing framework. All neural network operators requiring testing must pass through this entry point. Users only need to input one or more neural network operators to be tested, and the system will automatically parse them, select the corresponding registration interface for verification, and then call the corresponding execution interface. The execution interface will automatically match the corresponding parameters as input data for computation.

[0093] The registration layer consists of a series of registration configuration files for neural network operators. These configuration files contain the neural network operators that have already been implemented in the inference engine. For example, the configuration file for the neural network operator corresponding to the convolution operation registers operators with different precisions, data arrangements, and implementation platforms, and they are registered sequentially in the registration configuration files according to a sequential index.

[0094] The execution layer includes the execution process of each neural network operator, including input parameter extraction and parsing, input data transformation, test execution, output acquisition, and comparison of results with actual results. Each category of neural network operator uses the same standard test cases and the same input data.

[0095] The shared parameter layer consists of parameters shared by each type of neural network operator, i.e., operator instance templates, which are passed as data to the execution layer. Operator instance templates can be customized in unlimited quantities, allowing for comprehensive parameter definition to meet various testing scenarios. Shared parameters typically include the shape of the input data, operator attributes, etc., allowing developers to design parameters tailored to the characteristics of a specific operator category.

[0096] The common function layer contains some functions used by the testing framework, such as log printing interface, configuration file parsing interface, data conversion interface, performance result curve plotting interface, etc.

[0097] Based on the testing framework implemented above, the functions of each protocol layer are decoupled to the greatest extent possible, and the testing process is completed through mutual cooperation. Therefore, the various steps of the method of this application can be executed more efficiently. In practice, those skilled in the art can also flexibly modify the architecture based on the exemplary testing framework given above, as long as it is suitable for executing the various steps of the method of this application, and are not limited to the above examples.

[0098] As can be seen from the above embodiments, this application sets corresponding operator instance templates for neural network operators in deep learning models according to their categories. In each operator instance template, instance definition data corresponding to various operator instances describing that type of neural network operator is given. When it is necessary to test the neural network operator, the corresponding operator instance template is retrieved from the operator template library according to the type identifier of the neural network operator. The standard test cases corresponding to the type identifier are applied to perform unified testing on all operator instances defined by each instance definition data in the operator instance template. There is no need to apply the corresponding test cases for a single instance to perform testing, which improves testing efficiency, reduces the testing and deployment costs of deep learning models, avoids multi-source editing of test cases, and maximizes the stability and correctness of neural network operators. For application scenarios such as live streaming, which require the widespread use of deep learning models, it can achieve significant positive economic scale benefits.

[0099] Based on any embodiment of this application, please refer to Figure 3 Before responding to the operator test command and obtaining the type identifier of the corresponding neural network operator, the process includes:

[0100] Step S2100: Detect the code information of the neural network operator and determine the structural definition attributes of the neural network operator;

[0101] The code file of each neural network operator actually provides the definition information of the input and output parameters of each stage. The parameters defined by these definitions constrain the generation of operator instances of the neural network operator. Therefore, through these definitions, the structural definition attributes corresponding to the structure of the neural network operator can be known.

[0102] Specifically, by traversing and detecting the code information of the neural network operators, the definition information that serves as structural definition attributes is found. Based on this definition information, the variables that can define specific operator instances and their value ranges are obtained, thus forming the corresponding structural definition attributes. These structural definition attributes are then organized in an orderly manner to indicate the generation of corresponding instance definition data.

[0103] Step S2200: Obtain the standard definition data provided for each structural definition attribute, and construct the instance definition data of the neural network operator so that it contains the standard definition data corresponding to each result definition attribute;

[0104] In this application, standard definition data can be provided for various structural definition attributes that may be used by the neural network operator. These standard definition data can be understood as the initial values ​​that adapt to the corresponding structural definition attributes. Thus, after determining the various structural definition attributes of a neural network operator, the corresponding standard definition data of these structural definition attributes can be obtained and organized in an orderly manner to form the instance definition data of a corresponding operator instance of the neural network operator.

[0105] Of course, in a further embodiment, each structural definition attribute can provide multiple standard definition data. By cross-combining the multiple standard definition data corresponding to the multiple structural definition attributes, a large number of instance definition data can be derived, which is equivalent to defining a large number of operator instances.

[0106] Step S2300: Using the instance definition data as a template, derive other instance definition data by modifying some of the standard definition data with random numbers;

[0107] After generating instance definition data in the previous step, this data can be used as a template. Then, for any structural attribute in one of the instance definition data sets, the corresponding standard definition data can be replaced with a random value. This generates new instance definition data, as long as it doesn't duplicate other instance definition data. Generating new instance definition data with random numbers generalizes the differences between operator instances, allowing the neural network operator to be tested on randomly determined operator instances during testing. Introducing randomness into the generation of operator instances and then conducting tests helps to discover exceptions and anomalies during testing, helps to identify potential technical defects in the neural network operator, and ensures that the tested neural network operator is more robust.

[0108] Step S2400: Construct all instance definition data into operator instance templates of the neural network operator, and store them in the operator template library by mapping them to the type identifier of the neural network operator.

[0109] After determining multiple instance definition data of the neural network operator through the above process, all instance definition data are constructed into corresponding operator instance templates. For example, each instance definition data is vectorized, and the vector representations of all instance definition data are constructed into matrix representations, thus becoming the corresponding operator instance templates. After the construction of the operator instance templates of the neural network operator is completed, they are mapped and stored in the operator template library along with the type identifier of the neural network operator, thus completing the construction and configuration of the operator instance templates of the neural network operator.

[0110] As can be seen from the above embodiments, this application can automatically generate corresponding instance definition data by detecting the code information of the neural network operator, discovering its structural definition attributes, applying the standard definition data provided corresponding to the structural definition attributes, and further introducing random numbers to derive more instance definition data, automatically enriching the instance definition data of the operator instance, and then constructing all instance definition data into the operator instance template corresponding to the neural network operator, enriching the construction of the operator template library. Its execution process is automated and has high execution efficiency. Moreover, by introducing randomly generated instance definition data to generate random operator instances, compared with simply using standard definition data to define operator instances, the introduction of random factors plays a more positive role in testing neural network operators, helps to discover potential technical defects, and can improve the robustness of neural network operators that pass this test during the runtime phase.

[0111] Based on any embodiment of this application, please refer to Figure 4 After obtaining the execution metric data corresponding to each operator instance, including:

[0112] Step S3100: Compare the execution index data corresponding to each operator instance with the corresponding preset real index data, and calculate the test difference between the execution index data and the real index data of each operator instance.

[0113] When standard test cases are run for each operator instance of the neural network operator, and tests are performed on each operator instance through the standard test cases, and the corresponding execution index data is obtained, the execution index data obtained from the test can be measured according to the real index data given in advance.

[0114] The real-world metric data can be the metric data obtained by the corresponding neural network operator running in the development framework, and each specific data point generally corresponds one-to-one with the specific data points of the execution metric data. For example, the real-world metric data includes the execution time obtained by each operator instance of each neural network operator when running in the development framework. Similarly, the execution metric data can also include the execution time obtained by each operator instance under various backend configuration conditions. The real-world metric data can include the output data corresponding to each input data; similarly, the execution metric data also includes the output data obtained corresponding to the input data.

[0115] By measuring the difference between the performance metrics data obtained from the test and the preset real metrics data, the performance of the neural network operator under the corresponding backend configuration conditions can be examined, which is the purpose of the test. Typically, in one embodiment, the principle of comparative testing is applied to compare the performance metrics data with the real metrics data, determine the difference between the same type of data, and obtain the corresponding test difference, which can then serve as a measurement.

[0116] Step S3200: Select multiple operator instances whose test difference is less than a preset threshold as qualified operator instances, and obtain the instance definition data corresponding to the qualified operator instances;

[0117] To verify whether the performance of each operator instance of the neural network operator meets expectations and to select qualified operator instances, a corresponding threshold can be set for the test difference of each specific indicator data, i.e., the preset threshold. Then, the test difference is compared with its corresponding preset threshold. For example, when the test difference is related to the execution time, by comparing it with the preset threshold, the difference between the actual execution time and the execution time during testing can be determined. When this difference exceeds the preset threshold, it indicates that the corresponding operator instance does not achieve the expected effect and can therefore be rejected. When this difference is less than the preset threshold, it indicates that the test effect of the operator instance meets expectations and can be regarded as a qualified operator instance, and its instance definition data can be obtained accordingly.

[0118] Step S3300: Configure the neural network operator with the instance definition data of the qualified operator instance and use it for the deployment of the neural network model.

[0119] After determining the instance definition data of the qualified operator instances, one instance definition data can be automatically selected to configure the neural network operator. This allows the neural network operator to be implemented as the operator instance corresponding to the instance definition data, serving as a component of the neural network model and being deployed along with the neural network model. It is easy to understand that because the neural network operators used in the neural network model are configured according to operator instances with excellent test results, it ensures that the neural network model achieves relatively optimal performance after deployment.

[0120] In one embodiment, when selecting an operator instance from multiple qualified operator instances to configure the corresponding neural network operator in the neural network model, the multiple qualified operator instances can be further sorted according to the test difference corresponding to the obtained index data, and then the qualified operator instance with the smallest test difference can be selected as the best operator instance for configuring the corresponding neural network operator in the neural network model.

[0121] As can be seen from the above embodiments, after applying the standard test cases corresponding to the neural network operators and performing batch testing on multiple operator instances in the operator instance template corresponding to the neural network operators, the qualified operator instances whose test differences are less than a preset threshold are further determined based on the test difference between the execution indicator data obtained by each operator instance and its corresponding real indicator data. Then, the best qualified operator instance is selected to complete the configuration of the neural network model deployment stage. This can ensure that the corresponding neural network model obtains the best working performance. For platform scenarios such as live streaming, using a neural network model with high working performance to process various data in the platform, such as live images in live video streams, can save a lot of operating resources, improve service response speed, and improve user experience.

[0122] Based on any embodiment of this application, please refer to Figure 5 After obtaining the execution metric data corresponding to each operator instance, including:

[0123] Step S4100: Compare the execution index data corresponding to each operator instance with the corresponding preset real index data, and calculate the test difference between the execution index data and the real index data of each operator instance.

[0124] Similarly, when standard test cases are run for each operator instance of the neural network operator, and tests are performed on each operator instance through the standard test cases, and the corresponding execution index data is obtained, the execution index data obtained from the test can be measured based on the real index data given in advance.

[0125] The real-world metric data can be the metric data obtained by the corresponding neural network operator running in the development framework, and each specific data point generally corresponds one-to-one with the specific data points of the execution metric data. For example, the real-world metric data includes the execution time obtained by each operator instance of each neural network operator when running in the development framework. Similarly, the execution metric data can also include the execution time obtained by each operator instance under various backend configuration conditions. The real-world metric data can include the output data corresponding to each input data; similarly, the execution metric data also includes the output data obtained corresponding to the input data.

[0126] By measuring the difference between the performance metrics data obtained from the test and the preset real metrics data, the performance of the neural network operator under the corresponding backend configuration conditions can be examined, which is the purpose of the test. Typically, in one embodiment, the principle of comparative testing is applied to compare the performance metrics data with the real metrics data, determine the difference between the same type of data, and obtain the corresponding test difference, which can then serve as a measurement.

[0127] Step S4200: Identify operator instances whose test differences are greater than or equal to a preset threshold as abnormal operator instances;

[0128] In contrast to the screening principle for qualified operator instances in the previous embodiment, in this embodiment, considering the identification of abnormal operator instances in the operator instance template of the neural network model, after comparing the test difference with its corresponding preset threshold, operator instances with test differences greater than or equal to the preset threshold can be identified as abnormal operator instances.

[0129] Typically, when an abnormal operator instance appears in the operator instance template, it means that there are areas in the code information of the corresponding neural network operator that need improvement. Therefore, in one embodiment, alarm information corresponding to the abnormal operator instance can be generated. The alarm information includes instance definition data corresponding to the abnormal operator instance, and then the alarm information is directly displayed in the graphical user interface. A communication interface is then obtained to send the alarm information to the corresponding target user, such as the developer of the neural network operator. The communication interface can be an email address or an instant messaging interface.

[0130] Step S4300: Obtain instance definition data corresponding to the abnormal operator instance from the operator instance template, and adjust the instance definition data to derive multiple derived instance definition data;

[0131] In the presence of abnormal operator instances, considering the abundance of test results, it is possible to provide empirical data for troubleshooting the code information of neural network operators that exhibit abnormal operator instances. This allows relevant users to make comparisons and references based on sufficient empirical data in order to quickly identify technical defects. Therefore, based on the aforementioned abnormal operator instances, more instance definition data can be derived, and further testing can be conducted on the basis of these derived instance definition data to obtain more test results.

[0132] Therefore, in one embodiment, for any one anomaly operator instance, a fine-tuning method is used to slightly adjust the definition data corresponding to any one or more structural definition attributes. For example, the height of the input feature map corresponding to the convolution operation is adjusted from 32 to 31 or 33. This fine-tuning operation can be performed using a preset adjustment step size, which can be flexibly implemented by those skilled in the art. By fine-tuning at least one specific definition data in an anomaly operator instance, it is easy to understand that a corresponding derived instance data can be obtained. Similarly, based on the one or more of the aforementioned anomaly operator instances, a large amount of derived instance definition data can be obtained.

[0133] Step S4400: Call the standard test case to test the operator instance constructed based on the derived instance definition data and obtain the corresponding execution index data;

[0134] Based on the large number of derived instance definition data determined by fine-tuning the abnormal operator instances, the standard test cases corresponding to the neural network operators can be called again. By running the standard test cases again, the operator instances corresponding to each derived instance definition data can be tested, and the execution index data corresponding to these operator instances can be obtained, thereby enriching the test result data.

[0135] Step S4500: Based on the execution index data, find qualified operator instances in the operator instances derived from each abnormal operator instance, and construct a mapping relationship data between the qualified operator instance and the abnormal operator instance.

[0136] The execution metric data obtained after testing the corresponding operator instance in the derived instance definition data is checked accordingly. Specifically, following the method described earlier, the execution metric data is compared with the actual metric data of the abnormal operator instance corresponding to the derived instance definition data to determine the test difference. Whether the test difference is less than a preset threshold determines whether it is a qualified operator instance. This determines whether the operator instance in the fine-tuned derived instance definition data constitutes a qualified operator instance. When a qualified operator instance corresponding to an abnormal operator instance is found, the mapping relationship between the qualified operator instance and the abnormal operator instance is stored, displayed, or sent to the target user. It is easy to understand that the role of this mapping relationship data is to provide test users, developers, etc., with information about the differences in definition data between the abnormal operator instance and its nearest qualified operator instance. This provides more valuable reference information for troubleshooting neural network operator code information, helping to accelerate the debugging of neural network operators.

[0137] As can be seen from the above embodiments, this application addresses the situation where abnormal operator instances appear after testing multiple operator instances in the operator instance template of a neural network operator. By fine-tuning the definition data of these abnormal operator instances, the definition data of derived instances of these abnormal operator instances is obtained, thereby actually constructing more derived operator instances. Then, these derived operator instances are tested to find qualified operator instances. The qualified operator instances and their corresponding abnormal operator instances are then constructed into mapping relationship data to prompt the development, testing, and debugging of neural network operators, providing reliable reference information, further improving the success rate of technical problem diagnosis, accelerating the development, testing, and deployment of neural network models, shortening the development cycle of software products based on the neural network model, enabling platforms such as live streaming to achieve technology implementation more quickly, and improving the level of technical services and user experience.

[0138] Based on any embodiment of this application, please refer to Figure 6 After obtaining the execution metric data corresponding to each operator instance, including:

[0139] Step S5100: Obtain execution index data by testing each operator instance defined by the operator instance template of the same type of neural network operator under different backend configuration conditions. The backend configuration conditions are distinguished by at least one of the backend type identifier, operation precision level, and data arrangement type. The execution index data includes the execution time index of the corresponding operator instance.

[0140] Each neural network operator corresponds to different backend configuration conditions, and has its corresponding specific implementation in the deep learning inference engine. Therefore, when testing each neural network operator in this application, although the execution layer calls the specific implementation corresponding to each backend configuration condition for testing, the specific implementation of each backend configuration condition is still based on the operator instance template corresponding to the neural network operator to configure its various operator instances, and the standard test cases corresponding to the neural network operator are still used to execute the test process.

[0141] Based on this principle, the parameters specified when triggering the operator test command, such as the name of the neural network operator, typically include the type identifier of the neural network operator and its backend configuration conditions. Therefore, the specific implementation of the neural network operator can be determined based on these backend configuration conditions.

[0142] In some embodiments, the type identifier of the neural network operator can be specified separately in the operator test instruction without providing specific backend configuration conditions. Instead, the test framework of this application will automatically query the registered backend configuration conditions of the neural network operator from the registration configuration file and then obtain the corresponding specific implementation for each backend configuration condition to perform the test.

[0143] As mentioned earlier, each neural network operator can operate under different backend configurations. These different backend configurations can be distinguished by at least one of the following: backend type identifier, computational precision level, and data arrangement type. For example, for a neural network operator performing convolution operations, its backend type identifier can be arm v7, arm v8, metal, opengl, SSE, etc., corresponding to different chip hardware types; the computational precision level can be float32, float16, int16, int8, etc., corresponding to different computational precisions; and the data arrangement type can be nchw, nc4hw4, nc8hw8, etc., corresponding to common data arrangement methods in the industry. It is evident that neural network operators exhibit diversity in adapting to different backend configurations.

[0144] It is easy to understand that when the neural network operator is tested under the same standard test cases and the same operator instance template under different backend configuration conditions, the corresponding execution indicator data of the neural network operator under various backend configuration conditions can be obtained, such as the execution time indicator.

[0145] Step S5200: Based on the execution time index in the execution index data, calculate the average execution time corresponding to each backend configuration condition;

[0146] Each backend configuration condition obtains an execution time metric. Specifically, under each backend configuration condition, each operator instance in the same operator instance template can obtain its specific execution time metric determined after testing. Based on this, the average execution time metric corresponding to each operator instance can be calculated for each backend configuration condition as the average execution time. Thus, the average execution time of the neural network operator under different backend configuration conditions is obtained.

[0147] Step S5300: Generate a comparison chart showing the average execution time of different backend configuration conditions and their corresponding values, and display the comparison chart.

[0148] The average execution time of a neural network operator under different backend configurations is a key metric, effectively measuring its performance across various devices. This provides crucial reference for the development, testing, and deployment of neural network models. Therefore, in this embodiment, a comparison chart can be used to represent the average execution time of the neural network operator under different backend configurations. This chart can then be displayed in a graphical user interface or sent to the corresponding communication interface.

[0149] In one embodiment, the comparison chart can be represented by a bar chart, with the horizontal axis representing each backend configuration condition and the vertical axis representing the time information corresponding to the average execution time.

[0150] As can be seen from the above embodiments, the operator instance templates and standard test cases corresponding to the neural network operators of this application are applicable to different backend configuration conditions of the neural network operators. During the testing of the neural network operators, no manual intervention is required throughout the process, and key indicators such as the average execution time under various backend configuration conditions can be quickly obtained. These indicators are then quantified into visual comparison charts, which vividly present the test results under different backend configuration conditions. This not only significantly improves the testing efficiency of neural network operators but also optimizes the presentation of their test results information.

[0151] Before running the standard test cases corresponding to the neural network operator, based on any embodiment of this application, the following steps are included:

[0152] Step S6100: Determine the backend type identifier in the backend configuration conditions from the name of the neural network operator;

[0153] As mentioned earlier, the backend configuration conditions, particularly the backend identifier, can be included in the name of the neural network operator to make it easier to identify. When an operator test command is triggered, the backend type identifier in the corresponding neural network operator name can be determined. Of course, as mentioned earlier, the type identifier of the neural network operator itself will also be determined at the same time.

[0154] Step S6200: Query the registration configuration table to determine whether the backend type identifier has been pre-registered. If not, terminate the execution; otherwise, continue the execution.

[0155] As mentioned earlier, the backend configuration conditions for the neural network operators can be pre-registered in the registration configuration table. After the deep learning inference engine has added the corresponding specific implementation to its operator library, a registration record for the corresponding specific implementation is added to the registration configuration table. The registration configuration table can be stored as a file in formats such as CSV, JSON, and TXT, offering great flexibility. An exemplary description of the content of the registration configuration table is shown below:

[0156] 1,Conv2D_K1x1_S1x1_D1x1_G1_ARMv82_C4HW4_Half_M1

[0157] 2,Conv2D_K1x1_S1x1_D1x1_G1_ARMv82_C8HW8_Half_M1

[0158] 3,Conv2D_K3x3_S2x2_D1x1_G1_ARMv8_CHW_Float_M1

[0159] 4,Conv2D_K3x3_S2x2_D1x1_G1_ARMv8_CHW_Int16_M1

[0160] 5,Conv2d_G1_Img_C8HW8_F16_Metal_Fallback

[0161] ...

[0162] Taking each data entry as an example, Conv2D is the type identifier of the neural network operator, indicating that it belongs to the neural network operator of the operation type. ARMv82 is its backend type identifier, and it also includes data from other backend configuration conditions, which will not be elaborated here.

[0163] As can be seen from the registration configuration table in the example above, after determining the type identifier of the neural network operator itself and the backend type identifier in its backend configuration conditions, a query can be performed in the registration configuration table. When both the type identifier and the backend type identifier appear in the same record, i.e., when it has been registered, it means that the specific implementation of the neural network operator under the corresponding backend configuration conditions exists in the operator library. Thus, the subsequent business process of this application can continue through the verification of the corresponding operator test command. Otherwise, the specific implementation of the corresponding backend configuration conditions of the neural network operator is not registered in the registration configuration table, the verification fails, no further testing is required, and therefore, the execution of other subsequent business processes of this application can be terminated.

[0164] In this embodiment, the association and correspondence between different backend configuration conditions of various neural network operators are established through the registration configuration table. When starting the test of the neural network operator, the corresponding backend configuration conditions are verified first. Only after the verification is passed will the subsequent test process be executed. This avoids wasting the test framework's runtime resources, ensures that the test framework is more robust, and also ensures the data communication efficiency between the deep learning inference engine and the test framework implemented in this application.

[0165] Considering the application of the neural network operator testing method of this application to the live streaming scenario, a live streaming application method for neural network operators is provided to meet one of the purposes of this application. This method includes: using an image encoder to represent the image feature information of image frames in the live video stream of the live stream, wherein the image encoder includes the neural network operator obtained from the testing of the neural network operator testing method of any of the preceding embodiments; performing classification mapping based on the image feature information to determine the category information corresponding to the image frame; and controlling image occlusion processing of the image frame based on the category information.

[0166] In one embodiment, the category information can be the type of content involved in the live image. For example, after obtaining the image feature information, it is mapped to a classification space of a corresponding content classification system to obtain the classification probability corresponding to each category in the classification space. Each category in the classification space corresponds to each category in the content classification system, such as the content classification system including categories like {violation, compliance}. When the category information belongs to a certain category, the corresponding business logic is used to process the corresponding image frame accordingly. For example, when the category information belongs to the "violation" type, the corresponding image frame can be image masked by using a mosaic or blur layer to blur the image frame so that the corresponding image content cannot be clearly seen after the image frame is played; when the category information belongs to the "compliance" type, no processing is required.

[0167] Please see Figure 7 A neural network operator testing device provided to meet one of the purposes of this application includes an instruction response module 1100, a template query module 1200, and a test execution module 1300. The instruction response module 1100 is configured to respond to an operator test instruction and obtain a type identifier for the corresponding neural network operator. The template query module 1200 is configured to query an operator instance template corresponding to the type identifier from an operator template library. The operator instance template contains instance definition data corresponding to multiple operator instances of the neural network operator. The test execution module 1300 is configured to run standard test cases corresponding to the neural network operator, test each operator instance constructed according to the instance definition data, and obtain execution index data corresponding to each operator instance.

[0168] Based on any embodiment of this application, the neural network operator testing device of this application further includes: an attribute determination module, configured to detect the code information of the neural network operator and determine each structural definition attribute of the neural network operator; a basic definition module, configured to obtain standard definition data provided for each structural definition attribute and construct instance definition data of the neural network operator, such that it contains standard definition data corresponding to each result definition attribute; an instance derivation module, configured to use the instance definition data as a template and modify some of the standard definition data with random numbers to derive other instance definition data; and a template construction module, configured to construct all instance definition data into an operator instance template of the neural network operator and map it to the type identifier of the neural network operator and store it in the operator template library.

[0169] Based on any embodiment of this application, the neural network operator testing device of this application further includes: a difference determination module, configured to compare the execution index data corresponding to each operator instance with the corresponding preset real index data, and calculate the test difference between the execution index data and the real index data of each operator instance; a qualification screening module, configured to select multiple operator instances with test differences less than a preset threshold as qualified operator instances, and obtain the instance definition data corresponding to the qualified operator instances; and an operator configuration module, configured to configure the neural network operator with the instance definition data of the qualified operator instances, and use it for the deployment of the neural network model.

[0170] Based on any embodiment of this application, the neural network operator testing device of this application further includes: a difference determination module, configured to compare the execution index data corresponding to each operator instance with the corresponding preset real index data, and calculate the test difference between the execution index data and the real index data of each operator instance; an anomaly detection module, configured to determine the operator instance whose test difference is greater than or equal to a preset threshold as an abnormal operator instance; a fine-tuning derivation module, configured to obtain the instance definition data corresponding to the abnormal operator instance from the operator instance template, and adjust the instance definition data to derive multiple derived instance definition data; a derivation testing module, configured to call the standard test cases to test the operator instance constructed according to the derived instance definition data, and obtain the corresponding execution index data; and a derivation optimization module, configured to find a qualified operator instance in each operator instance derived from the abnormal operator instance based on the execution index data, and construct a mapping relationship data between the qualified operator instance and the abnormal operator instance.

[0171] Based on any embodiment of this application, the neural network operator testing device of this application further includes: an index acquisition module, configured to acquire execution index data obtained by performing tests on various operator instances defined by the operator instance template of neural network operators with the same type identifier under different backend configuration conditions, wherein the backend configuration conditions are distinguished by at least one of backend type identifier, operation precision level, and data arrangement type, and the execution index data includes the execution time index of the corresponding operator instance; a time statistics module, configured to calculate the average execution time corresponding to each backend configuration condition based on the execution time index in the execution index data; and a chart generation module, configured to generate a comparison chart representing different backend configuration conditions and their corresponding average execution time, and display the comparison chart.

[0172] Based on any embodiment of this application, the neural network operator testing device of this application further includes: a backend determination module, configured to determine the backend type identifier in the backend configuration conditions from the name of the neural network operator; and a registration query module, configured to query the registration configuration table to determine whether the backend type identifier has been pre-registered. If it has not been registered, execution is terminated; otherwise, execution continues.

[0173] To address the aforementioned technical problems, embodiments of this application also provide an electronic device. For example... Figure 8 The diagram shows the internal structure of an electronic device. This electronic device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. The computer-readable storage medium stores an operating system, a database, and computer-readable instructions. The database may store control information sequences. When the computer-readable instructions are executed by the processor, they enable the processor to implement a neural network operator testing method. The processor provides computational and control capabilities to support the operation of the entire electronic device. The memory stores computer-readable instructions, which, when executed by the processor, enable the processor to execute the neural network operator testing method of this application. The network interface of the electronic device is used for communication with a terminal. Those skilled in the art will understand that… Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0174] In this embodiment, the processor is used to execute... Figure 7 The system contains the specific functions of each module and its sub-modules, and the memory stores the program code and various data required to execute these modules or sub-modules. The network interface is used for data transmission between the user terminal and the server. In this embodiment, the memory stores the program code and data required to execute all modules / sub-modules in the neural network operator testing device of this application, and the server can call the server's program code and data to execute the functions of all sub-modules.

[0175] This application also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the neural network operator testing method of any embodiment of this application.

[0176] This application also provides a computer program product, including a computer program / instructions that, when executed by one or more processors, implement the steps of the method described in any embodiment of this application.

[0177] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0178] In summary, this application retrieves the corresponding operator instance template from the operator template library based on the type identifier of the neural network operator, applies standard test cases corresponding to the type identifier, and performs unified testing on all operator instances defined by the instance definition data in the operator instance template. This results in higher testing efficiency, reduces the testing and deployment costs of deep learning models, avoids multi-source editing of test cases, and maximizes the stability and correctness of neural network operators. For application scenarios such as live streaming that require the widespread use of deep learning models, this approach can achieve significant positive economic benefits.

Claims

1. A method of testing a neural network operator, the method comprising: The method comprises the following steps: In response to the operator test instruction, the type identifier of the neural network operator is obtained from the name of the corresponding neural network operator, and the type identifier is used to indicate the operation type to which the neural network operator belongs; An operator instance template corresponding to the type identifier is queried from the operator template library, and the operator instance template contains instance definition data corresponding to a plurality of operator instances of the neural network operator. The instance definition data of each operator instance is represented in the form of a vector; A standard test case corresponding to the neural network operator is run, and each operator instance constructed according to the instance definition data is tested respectively to obtain execution index data corresponding to each operator instance, wherein: The standard test case is obtained by standardizing and abstracting the test code of various possible operator instances of the neural network operator, so that there is only one standard test case corresponding to each neural network operator; by executing the standard test case, each operator instance of the neural network operator is identified by the test code of the standard test case, and then the corresponding test business logic is called to implement corresponding tests on the corresponding operator instance; When the standard test case of the neural network operator is run, each instance definition data in the operator instance template of the neural network operator is transmitted to an execution unit in a deep learning inference engine, and the execution unit is controlled to configure the specific implementation of the neural network operator by using the instance definition data, so as to obtain the corresponding operator instance. Then, by using the preset test data, the operator instance is inputted to run, and the corresponding result data is obtained and returned.

2. The neural network operator testing method of claim 1, wherein, Before obtaining the type identifier of the neural network operator from the name of the corresponding neural network operator in response to the operator test instruction, the following steps are included: Detecting the code information of the neural network operator to determine the structure definition attributes of the neural network operator; Obtaining the standard definition data provided for each structure definition attribute to construct the instance definition data of the neural network operator, so that it contains the standard definition data corresponding to each structure definition attribute; Using the instance definition data as a template, other instance definition data is derived by modifying part of the standard definition data in the instance definition data with random numbers; All instance definition data is constructed into the operator instance template of the neural network operator, and is stored in the operator template library in correspondence with the type identifier of the neural network operator.

3. The neural network operator testing method of claim 1, wherein, After obtaining the execution index data corresponding to each operator instance, the following steps are included: Comparing the execution index data corresponding to each operator instance with the preset real index data to calculate the test difference value between the execution index data and the real index data of each operator instance; Selecting a plurality of operator instances with a test difference value less than a preset threshold as qualified operator instances, and obtaining the instance definition data corresponding to the qualified operator instances; The instance definition data of the qualified operator instances is used to configure the neural network operator, which is used for deployment of a neural network model.

4. The neural network operator testing method of claim 1, wherein, After obtaining the execution index data corresponding to each operator instance, the following steps are included: comparing the execution indicator data corresponding to each operator instance with the corresponding preset real indicator data, and calculating a test difference value between the execution indicator data and the real indicator data of each operator instance; determining an operator instance with a test difference value greater than or equal to a preset threshold as an abnormal operator instance; obtaining instance definition data corresponding to the abnormal operator instance from the operator instance template, and adjusting the instance definition data to derive a plurality of derived instance definition data; calling the standard test case to test an operator instance constructed according to the derived instance definition data, and obtaining corresponding execution indicator data; based on the execution indicator data, finding a qualified operator instance in each operator instance derived from the abnormal operator instance, and forming mapping relationship data between the qualified operator instance and the abnormal operator instance.

5. The method of claim 1, wherein, After obtaining the execution indicator data corresponding to each operator instance, the method comprises: obtaining execution indicator data obtained by testing each operator instance defined by an operator instance template of a neural network operator of the same type under different backend configuration conditions, the backend configuration conditions being distinguished by at least one of a backend type identifier, an operation precision level, and a data arrangement type, and the execution indicator data comprising an execution time indicator of the corresponding operator instance; based on the execution time indicator in the execution indicator data, statistically calculating an average execution time corresponding to each backend configuration condition; generating a comparison chart representing different backend configuration conditions and their corresponding average execution times, and displaying the comparison chart.

6. The neural network operator testing method according to any one of claims 1 to 5, characterized in that, Before running the standard test case corresponding to the neural network operator, the method comprises: determining a backend type identifier in the backend configuration condition from the name of the neural network operator; querying a registration configuration table to determine whether the backend type identifier has been pre-registered, and terminating execution when it has not been registered, otherwise continuing execution.

7. A method for live application of neural network operators, characterized in that, The method comprises: using an image encoder to represent image feature information of an image frame in a live video stream of a network live broadcast, the image encoder comprising a neural network operator tested by the method of any one of claims 1 to 6; performing classification mapping according to the image feature information to determine category information corresponding to the image frame.

8. A neural network operator testing apparatus, characterized by, The method comprises: an instruction response module configured to obtain a type identifier of the neural network operator from the name of the corresponding neural network operator in response to an operator test instruction; a template query module configured to query an operator instance template corresponding to the type identifier from an operator template library, the operator instance template comprising instance definition data corresponding to a plurality of operator instances of the neural network operator, and the instance definition data of each operator instance being represented in the form of a vector; a test execution module configured to run a standard test case corresponding to the neural network operator, and test each operator instance constructed according to the instance definition data to obtain execution indicator data corresponding to each operator instance, wherein: The standard test case is based on standardized abstraction of test codes for various possible operator instances of the neural network operator, so that each neural network operator has only one standard test case corresponding thereto; by executing the standard test case, each operator instance of the neural network operator is identified by the test code of the standard test case, and then corresponding test business logic is called to implement corresponding test on the corresponding operator instance; During runtime of the standard test case of the neural network operator, each instance definition data in the operator instance template of the neural network operator is transmitted to an execution unit in a deep learning inference engine, the execution unit is controlled to configure a specific implementation of the neural network operator by using the instance definition data, so as to obtain a corresponding operator instance, and then, by using preset test data, the operator instance is input for running, corresponding result data is obtained and returned.

9. An electronic device comprising a central processing unit and a memory, characterized in that The central processing unit is configured to call and run a computer program stored in the memory to perform the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer program is stored in the form of computer readable instructions and is implemented according to the method of any one of claims 1 to 7, and when the computer program is called and run by a computer, the steps included in the corresponding method are performed.

Images