Test method and system of neural network compiler, storage medium, program product and electronic device

Abstract

The present application relates to a kind of neural network compiler's test method and system, storage medium, program product and electronic equipment.The test optimization method includes: obtaining neural network model;Original computational graph of neural network model is generated and semantic standardization preprocessing is carried out, and intermediate computational graph is obtained;Intermediate computational graph is decomposed into multiple computing subgraphs;The multi-level feature of each computing subgraph is extracted, and the feature identification of computing subgraph is generated based on multi-level feature;Multi-level matching search is carried out in historical test record database based on feature identification;The compiled deduplication evaluation of the computing subgraph matched is carried out;Determine test decision based on evaluation result;According to test decision, corresponding test operation is carried out to the neural network compiler to be tested, to obtain test result.The technical scheme of the present application can guarantee test coverage, avoid the problem of missing test while effectively reducing redundant test, reduce test resource consumption, improve overall test efficiency.

Description

Technical Field

[0001] This invention relates to the field of compiler testing technology, specifically to a testing method and system for a neural network compiler, a storage medium, a program product, and an electronic device. Background Technology

[0002] With the popularization of artificial intelligence technology, neural network models have been widely applied to various terminal devices and computing platforms. Neural network compilers are the core tools connecting neural network models to the underlying hardware, and the correctness and operational stability of their compilation functions need to be verified through extensive testing. Existing neural network compiler testing schemes struggle to efficiently identify semantically equivalent computational subgraphs and cannot reliably determine subgraphs that can skip tests, easily leading to test redundancy. Summary of the Invention

[0003] To address the technical problems of existing neural network compiler testing schemes, such as difficulty in efficiently identifying semantically equivalent computational subgraphs and the inability to reliably determine subgraphs that can be skipped, which easily leads to test redundancy, this invention provides a testing method and system for neural network compilers, a storage medium, a program product, and an electronic device. The technical solution of this invention can effectively reduce redundant tests, lower test resource consumption, and improve overall testing efficiency while ensuring test coverage.

[0004] To achieve the above objectives, a first aspect of the present invention provides a method for testing a neural network compiler, comprising: Obtain the neural network model as the test case for the compiler of the neural network to be tested; The original computation graph of the neural network model is generated, and the original computation graph is subjected to semantic normalization preprocessing to obtain an intermediate computation graph, wherein the semantically equivalent intermediate computation graphs have the same structured feature sequence. The intermediate computation graph is decomposed into multiple computation subgraphs, wherein each computation subgraph is a semantically complete computation subgraph; Extract multi-level features from each computational subgraph and generate feature identifiers for the computational subgraph based on the multi-level features, wherein the multi-level features include topological features, operator features, and data flow features; Multi-level matching retrieval is performed in the historical test record database based on the feature identifiers, wherein the historical test record database stores feature identifiers of historical calculation subgraphs and related information of historical test records associated with the feature identifiers of the historical calculation subgraphs; The computational subgraph that retrieves matching historical test record information from the historical test record database is compiled, deduplicated, and evaluated to obtain an evaluation result. The evaluation result characterizes the similarity between the computational subgraph and the matching historical computational subgraph and the risk of skipping the test. A testing decision is determined based on the evaluation results, wherein the testing decision includes one of executing a full test, executing a downgraded test, or skipping the test; Based on the test decision, corresponding test operations are performed on the compiler of the neural network to be tested to obtain test results. The test operations include hierarchical compilation and verification of the neural network model.

[0005] In one possible implementation of the first aspect, the semantic normalization preprocessing of the original computation graph includes at least one of the following operations: Perform a standardized naming operation on all operator nodes in the original computation graph to unify the names of operator nodes with the same function; Perform a pre-computation operation on the operator nodes in the original computation graph that only take constants as input, and replace the operator nodes that only take constants as input with the pre-computed result values; Deletion operations are performed on operator nodes and connecting edges in the original computation graph whose execution results do not affect the final output of the original computation graph; and Based on the dependencies of the computational logic, the topological order of all nodes in the original computation graph is determined, so that semantically equivalent original computation graphs have a consistent node arrangement order.

[0006] In one possible implementation of the first aspect, the topological features include at least one of the following: the number of nodes in the computational subgraph, the connectivity between nodes, the depth of the computational subgraph, and the width of the computational subgraph.

[0007] In one possible implementation of the first aspect, the operator features include the operator type and / or parameters of each operator node in the computation subgraph.

[0008] In one possible implementation of the first aspect, the data flow characteristics include at least one of the shape, data type, and memory layout information of the tensors in the computation subgraph.

[0009] In one possible implementation of the first aspect, the feature identifier of the computational subgraph includes a first-level hash value, a second-level hash value, and a third-level hash value; The first-level hash value is generated based on the topological features; the second-level hash value is generated based on the topological features and the operator features; and the third-level hash value is generated based on the topological features, the operator features, and the data flow features.

[0010] In one possible implementation of the first aspect, the method further includes: For the current computational subgraph, a third-level match is performed in the historical test record database based on the third-level hash value. If the third-level match fails, a second-level match is performed in the historical test record database based on the second-level hash value. If the second-level match fails, a first-level match is performed in the historical test record database based on the first-level hash value. Therefore, this hash-value-based hierarchical matching method can efficiently match the current computational subgraph in the historical test record database. Figure 1 Historical computational subgraphs with similarity or high degree of similarity.

[0011] In one possible implementation of the first aspect, the historical test record information includes at least one of the following: multi-level feature information of the historical computation subgraph, test result information of the historical computation subgraph, and tested parameter range information of the historical computation subgraph.

[0012] In one possible implementation of the first aspect, the test result information of the historical computation subgraph includes at least one of test status information, test execution time, and test platform information, wherein the test status information characterizes whether the test has passed.

[0013] In one possible implementation of the first aspect, the compilation and deduplication evaluation of the computational subgraph includes: The computational subgraph is evaluated for compilation and deduplication based on at least one of the following dimensions: topological difference dimension, parameter difference dimension, combinatorial difference dimension, and test risk dimension.

[0014] In one possible implementation of the first aspect, the evaluation result is calculated using the following formula: Evaluation result = w1 Topological difference dimension value + w2 Parameter difference dimension value + w3 Combined difference dimension values ​​+ w4 The test risk dimension values ​​are defined, where w1, w2, w3, and w4 are the weight coefficients for the corresponding dimension values. The weight coefficients w1, w2, w3, and w4 are dynamically adjusted based on historical test data in the historical test record database.

[0015] In one possible implementation of the first aspect, the test decision is determined based on the evaluation results in the following manner: When the evaluation result is greater than or equal to the first decision threshold, the test decision is determined to execute the complete test; When the evaluation result is greater than or equal to the second decision threshold and less than the first decision threshold, the test decision is determined to be to perform a degradation test. When the evaluation result is less than the second decision threshold, the test decision is determined to be to skip the test; Wherein, the first decision threshold is greater than the second decision threshold.

[0016] In one possible implementation of the first aspect, after performing the corresponding test operations on the compiler of the neural network under test according to the test decision, the method further includes: updating the historical test record database in the following manner: Store the feature identifier and multi-level features of the current computation subgraph into the historical test record database; or store the test result information and test execution information of the current computation subgraph into the historical test record database; or update the tested parameter range information corresponding to the current computation subgraph in the historical test record database.

[0017] In one possible implementation of the first aspect, after updating the historical test record database, the method further includes: If the test status information of the current computation subgraph is "test failed", then the model used to calculate the test risk dimension value is adjusted according to the failure mode of this test.

[0018] In one possible implementation of the first aspect, the historical test record database adopts a hierarchical storage architecture: the first layer is a hash index layer, used to store the association mapping relationship between the feature identifier and the historical computation subgraph; the second layer is a vector storage layer, used to store the graph embedding vector corresponding to the historical computation subgraph; and the third layer is a structured data layer, used to store the multi-level feature information, test result information and tested parameter range information of the historical computation subgraph.

[0019] Secondly, the present invention provides a testing system for a neural network compiler, comprising: The test case acquisition module is configured to acquire the neural network model that serves as the test case for the neural network compiler under test. The computation graph generation and preprocessing module is configured to generate the original computation graph of the neural network model and perform semantic normalization preprocessing on the original computation graph to obtain an intermediate computation graph, wherein the semantically equivalent intermediate computation graphs have the same structured feature sequence. The computation graph decomposition module is configured to decompose the intermediate computation graph into multiple computation subgraphs, wherein each computation subgraph is a semantically complete computation subgraph; The feature extraction and label generation module is configured to extract multi-level features of each computational subgraph and generate feature labels of the computational subgraph based on the multi-level features, wherein the multi-level features include topological features, operator features and data flow features; The matching and retrieval module is configured to perform multi-level matching and retrieval in the historical test record database based on the feature identifier, wherein the historical test record database stores the feature identifier of the historical calculation subgraph and related information of the historical test record associated with the feature identifier of the historical calculation subgraph. The deduplication evaluation module is configured to compile and deduplicate the computational subgraph that retrieves matching historical test record information from the historical test record database to obtain an evaluation result, wherein the evaluation result characterizes the similarity between the computational subgraph and the matched historical computational subgraph and the risk of skipping the test; The decision-making module is configured to determine a test decision based on the evaluation results, wherein the test decision includes one of executing a full test, executing a downgraded test, and skipping the test; The decision execution module is configured to perform corresponding test operations on the neural network compiler under test according to the test decision, so as to obtain test results. The test operations include hierarchical compilation and verification of the neural network model.

[0020] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the executable computer program in the storage medium is executed by a processor, it is capable of implementing a test method for a neural network compiler as described in the first aspect and any possible implementation thereof.

[0021] Fourthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements a testing method for a neural network compiler as described in the first aspect and any possible implementation thereof.

[0022] Fifthly, the present invention provides an electronic device including a memory and a processor, the memory being used to store a computer program executable by the processor; the processor being used to execute the computer program in the memory to implement a test method for a neural network compiler as described in the first aspect and any possible implementation thereof.

[0023] Compared with the prior art, the beneficial effects of the present invention are as follows: The technical solution of this invention preprocesses the original computation graph through semantic normalization to obtain an intermediate computation graph. Semantically equivalent intermediate computation graphs share the same structured feature sequence, providing a unified representation basis for computation graph comparison. Based on this, feature identifiers for computation subgraphs are generated. Since these identifiers are generated based on the extracted multi-level features of each computation subgraph, they carry features from multiple levels. Therefore, multi-level matching retrieval is performed in the historical test record database based on these feature identifiers, enabling efficient and accurate location of historical test records similar to the current computation subgraph. Simultaneously, evaluation results are obtained by compiling and deduplicating the computation subgraphs that have been matched with historical records. Since the evaluation results characterize the similarity between the current computation subgraph and the matched historical computation subgraph, as well as the risk of skipping tests, the similarity between the current computation subgraph and historical records, and the risk of skipping tests, can be quantified based on the evaluation results. Based on these evaluation results, corresponding test decisions are determined, and corresponding test operations are executed. This achieves accurate identification and elimination of redundant test cases while maintaining test coverage, improving the efficiency of test resource utilization. Attached Figure Description

[0024] Figure 1 A test scenario for a neural network compiler is illustrated according to an exemplary embodiment.

[0025] Figure 2 A flowchart of a testing method for a neural network compiler is shown according to an exemplary embodiment.

[0026] Figure 3 A block diagram of a test system for a neural network compiler is shown according to an exemplary embodiment.

[0027] Figure 4 A block diagram of an electronic device is shown according to an exemplary embodiment. Detailed Implementation

[0028] Unless otherwise defined, the technical or scientific terms used in this specification and claims shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. Specific embodiments of the invention will be described below with reference to the accompanying drawings. It should be noted that, in order to provide a concise description, this specification cannot provide a detailed description of all features of the actual embodiments. Without departing from the spirit and scope of the invention, those skilled in the art can make modifications and substitutions to the embodiments of the invention, and the resulting embodiments are also within the protection scope of the invention.

[0029] Neural network compiler testing schemes in related technologies are prone to technical problems such as test redundancy and missed tests because they are difficult to efficiently identify semantically equivalent computational subgraphs and cannot reliably determine subgraphs that can be skipped.

[0030] To address the aforementioned technical problems, this invention provides a testing method and system for neural network compilers, a storage medium, a program product, and an electronic device. The technical solution of this invention can effectively reduce redundant tests, decrease test resource consumption, and improve overall testing efficiency while ensuring test coverage.

[0031] The following detailed description of some embodiments of the present invention is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0032] To facilitate understanding of the technical solution of this invention, firstly, in conjunction with... Figure 1 The application scenarios shown illustrate the technical solution of the present invention.

[0033] exist Figure 1 In the application scenario shown, a neural network model is used as the test case input to the test system 100 of the neural network compiler. After obtaining the neural network model, the test system 100 performs semantic normalization preprocessing on the original computation graph of the neural network model to obtain an intermediate computation graph. Semantically equivalent intermediate computation graphs have the same structured feature sequence, which can establish a unified representation basis for comparison of computation graphs. Based on this, feature identifiers for computation subgraphs are generated. Since these feature identifiers are generated based on the extracted multi-level features of each computation subgraph, they carry features from multiple levels of the computation subgraph. Therefore, multi-level matching retrieval is performed in the historical test record database based on the feature identifiers, enabling efficient and accurate location of historical test records similar to the current computation subgraph. Simultaneously, by performing compilation and deduplication evaluation on the computation subgraphs that have retrieved matching historical information, an evaluation result is obtained. Since the evaluation result characterizes the similarity between the current computation subgraph and the matched historical computation subgraph, as well as the risk of skipping tests, the similarity between the current computation subgraph and the historical record, and the risk of skipping tests, can be quantified based on the obtained evaluation result. Based on this, corresponding testing decisions are made according to the evaluation results, and corresponding testing operations are executed. This ensures accurate identification and elimination of redundant test cases while maintaining test coverage, thereby improving the efficiency of test resource utilization.

[0034] Finally, the test results are fed back to the R&D personnel through the terminal. Because the test results obtained by the technical solution of this invention not only maintain complete test coverage and achieve accurate test risk quantification, but also eliminate invalid data caused by redundant test links, the R&D personnel can quickly and accurately complete the test verification and defect investigation of the neural network compiler under test, shorten the test cycle, and reduce test resource consumption.

[0035] The computational subgraph refers to the set of topological structures of corresponding independent computational units obtained by decomposing the complete computational graph of the neural network model under test. If two intermediate computational graphs have the same computational logic, data flow, executable code generated by the compiler, and test results, then these two intermediate computational graphs can be called semantically equivalent intermediate computational graphs.

[0036] It should be noted that, Figure 1 The application scenario shown is merely an exemplary application scenario of the technical solution of this invention and is not intended to limit the application scenario of this invention. The testing method for neural network compilers proposed in this invention is universally applicable to automated testing scenarios of neural network compilers in various architectures and deployment environments, and can adapt to different testing devices, different database deployment forms, and different types of neural network compilation testing tasks under test.

[0037] The following will combine Figure 2 The flowchart shown provides a detailed description of a testing method for a neural network compiler provided by this invention. (Reference) Figure 2 The present invention provides a testing method for a neural network compiler, comprising the following steps S11 to S18: Step S11: Obtain the neural network model as the test case for the neural network compiler to be tested.

[0038] Step S12: Generate the original computation graph of the neural network model, and perform semantic normalization preprocessing on the original computation graph to obtain the intermediate computation graph. Among them, the semantically equivalent intermediate computation graphs have the same structured feature sequence.

[0039] In some embodiments, generating the original computational graph of the neural network model can be achieved by calling a compiler front-end import tool corresponding to the model format. The compiler front-end is responsible for parsing the neural network model file saved in a specific format and converting its internal computational logic and structure into a unified intermediate representation (IR) defined by the compiler of the neural network under test. The initial graph structure obtained from this conversion process, without subsequent semantic normalization, is the original computational graph. This ensures that neural network models exported from different sources or frameworks as test cases can be converted into a format that can be consistently processed in subsequent steps. This allows semantically equivalent computational graphs, after preprocessing, to present the same sequence of structured features, thus establishing a reliable and consistent benchmark for subsequent accurate matching and similarity comparison based on graph structure features. This fundamentally solves the problem of being unable to identify semantically equivalent subgraphs due to differences in surface form.

[0040] Step S13: Decompose the intermediate computation graph into multiple computation subgraphs, where each computation subgraph is a semantically complete computation subgraph.

[0041] The multiple computational subgraphs refer to at least two computational subgraphs. The number of computational subgraphs obtained from the decomposition can be determined based on the network structure complexity of the neural network model used as the test case of the neural network compiler under test and the set partitioning rules. The partitioning rules can be configured according to specific test requirements. For example, for a neural network model with a specific architecture, the intermediate computational graph can be divided according to the granularity of the network layer, or it can be further divided according to the granularity of the functional modules, with the self-attention computation module and the feedforward network module as independent computational subgraphs.

[0042] It's easy to understand that since each computational subgraph corresponds to an independent computational unit, using semantically complete computational subgraphs as the basic analysis unit allows for finer-grained testing, improving the accuracy of deduplication. It also provides a basis for subsequent differentiated testing decisions for different computational subgraphs, facilitating fine-grained scheduling of test resources.

[0043] Step S14: Extract multi-level features of each computational subgraph and generate feature identifiers of the computational subgraph based on the multi-level features, wherein the multi-level features include topological features, operator features and data flow features.

[0044] It is easy to understand that the multi-level features specifically consist of three levels of features, which are arranged in order of increasing granularity: topological features, operator features, and data flow features. Specifically, the first level of features are topological features, used to represent the topological structure information of the computational subgraph; the second level of features are operator features, used to represent the computational logic attributes within the computational subgraph; and the third level of features are data flow features, used to represent the data transmission and storage attributes of the computational subgraph.

[0045] Therefore, subsequent fast matching and retrieval based on feature identifiers can be achieved. Furthermore, since the feature identifiers of the computational subgraph include multi-level features of the computational subgraph, more comprehensive and accurate retrieval can be achieved.

[0046] Step S15: Perform multi-level matching retrieval in the historical test record database based on feature identifiers. The historical test record database stores feature identifiers of historical calculation subgraphs and related information of historical test records associated with the feature identifiers of historical calculation subgraphs.

[0047] Step S16: Compile and deduplicate the computational subgraph that retrieves matching historical test record information from the historical test record database to obtain the evaluation result. The evaluation result represents the similarity between the computational subgraph and the matched historical computational subgraph and the risk of skipping the test.

[0048] This allows for precise quantification of the similarity between the current subgraph and historical subgraphs, as well as the potential risk of skipping the current test. It overcomes the shortcomings of traditional testing methods that cannot quantify test risks and rely solely on subjective judgment based on experience to determine whether to skip a test, thus eliminating the risk of missed tests and redundant testing issues arising from subjective judgment.

[0049] Step S17: Determine a testing decision based on the evaluation results, wherein the testing decision includes one of executing a full test, executing a downgraded test, or skipping the test.

[0050] Establishing a tiered testing decision-making mechanism based on quantitative evaluation results enables precise matching of test intensity with test risk and the degree of difference in computational subgraphs. Thus, a complete testing process is retained for high-risk, highly differentiated computational subgraphs, avoiding missed compilation defects. For medium-risk, partially differentiated computational subgraphs, a simplified testing process can be implemented using downgraded testing. For low-risk, highly similar computational subgraphs, redundant tests can be skipped. This effectively solves the problem of redundant testing or missed risk detection caused by undifferentiated testing in traditional testing schemes, ensuring both testing reliability and high testing efficiency.

[0051] Step S18: Execute corresponding test operations on the compiler of the neural network to be tested according to the test decision to obtain test results. The test operations include hierarchical compilation and verification of the neural network model.

[0052] This can significantly reduce unnecessary compilation and verification workload, reduce testing time and hardware resource consumption, effectively optimize the overall testing process, and improve the testing efficiency and test resource utilization of neural network compilers.

[0053] It is understood that the execution order of steps S11 to S18 above is only an illustration. In other embodiments, other execution orders may be used, and some steps may be split or combined. This is not limited here.

[0054] In some embodiments, the semantic normalization preprocessing of the original computation graph includes at least one of the following operations: performing a standardized naming operation on all operator nodes in the original computation graph to unify the names of operator nodes with the same function; performing a pre-computation operation on operator nodes in the original computation graph that only take constants as input, replacing the operator nodes that only take constants as input with the pre-computed result value, and deleting the operator nodes that take constants as input; performing a deletion operation on operator nodes and connecting edges in the original computation graph whose execution results do not affect the final output of the original computation graph; and determining the topological order of all nodes in the original computation graph according to the dependencies of the computational logic, so that semantically equivalent original computation graphs have a consistent node arrangement order. Therefore, by performing semantic normalization processing on all operator nodes in the original computation graph, the differences in operator naming, redundant computation, invalid structures, and node arrangement of the original computation graph can be eliminated, enabling semantically equivalent original computation graphs to form a completely consistent structured feature sequence, achieving a unified representation of different forms of original computation graphs. This approach solves the technical problem that semantically equivalent computation graphs cannot be identified due to structural differences in traditional schemes. It can avoid the structural representation bias problem in the preprocessing stage and provide a unified comparison benchmark for feature extraction, similarity comparison, and multi-level matching retrieval of subsequent computational subgraphs. This helps to improve the accuracy of semantically equivalent subgraph identification and reduce the probability of misjudgment and missed judgment in computational subgraph matching.

[0055] In some embodiments, the topological features include at least one of the following: the number of nodes in the computational subgraph, the connectivity between nodes, the depth of the computational subgraph, and the width of the computational subgraph. In some embodiments, the topological features include one of the following: the number of nodes in the computational subgraph, the connectivity between nodes, the depth of the computational subgraph, and the width of the computational subgraph. In some embodiments, the topological features include two of the following: the number of nodes in the computational subgraph, the connectivity between nodes, the depth of the computational subgraph, and the width of the computational subgraph. In some other embodiments, the topological features include three of the following: the number of nodes in the computational subgraph, the connectivity between nodes, the depth of the computational subgraph, and the width of the computational subgraph.

[0056] By using the number of nodes, the connectivity between nodes, and the depth and width of the computational subgraph as dimensions for representing topological features, the overall topological structure attributes of the computational subgraph can be comprehensively and accurately quantified, fully covering the skeletal structural features of the computational subgraph. This allows for precise differentiation of computational subgraphs with different topologies, avoiding problems such as inaccurate or incomplete matching due to missing topological features.

[0057] In some embodiments, the operator features include the operator type and / or the parameters of each operator node in the computation subgraph. In some embodiments, the operator features include the operator type of each operator node in the computation subgraph. In some embodiments, the operator features include the parameters of each operator node in the computation subgraph. In other embodiments, the operator features include the operator type and the parameters of each operator node in the computation subgraph. The parameters of the operator node are the inherent configuration parameters and dimension operation parameters of each operator node, which are quantitative feature parameters that distinguish them from the operator type. They are used to characterize the differentiated configuration information of operator nodes of the same type, specifically including the model dimension parameters corresponding to the operator, the operator-specific configuration hyperparameters, and the operator operation constraint parameters. These are the quantitative basis for calculating parameter difference dimensions, generating second-level hash values, and subsequently evaluating the similarity of computation subgraphs in this invention.

[0058] By extracting the operator type and / or operator parameters of each operator node within the computational subgraph as operator features, the attribute details of operators within the computational subgraph can be accurately characterized, fully covering the computational attribute information at the operator level. This avoids the limitation of a single topological feature failing to characterize the specific computational attributes of an operator, enriches the semantic representation dimension of the computational subgraph, and provides a feature basis for subsequent second-level hash value generation and refined filtering and matching at the operator parameter dimension. This effectively improves the ability to identify subtle differences between similar computational subgraphs.

[0059] In some embodiments, data flow features include at least one of the shape, data type, and memory layout information of tensors in the computation subgraph. By using the shape, data type, and memory layout information of tensors as data flow features, the data transmission and storage attributes during the operation of the computation subgraph can be accurately characterized.

[0060] In some embodiments, the feature identifiers of the computed subgraph include a first-level hash value, a second-level hash value, and a third-level hash value; wherein, the first-level hash value is generated based on topological features; the second-level hash value is generated based on topological features and operator features; and the third-level hash value is generated based on topological features, operator features, and data flow features. Since hash values ​​at different levels can be specifically matched to the subgraph difference filtering requirements of different dimensions, retrieval efficiency can be guaranteed while maintaining retrieval accuracy.

[0061] In some embodiments, the testing method for the neural network compiler provided in this application further includes: for the current computational subgraph, performing a third-level match in the historical test record database based on the third-level hash value; if the third-level match fails, performing a second-level match in the historical test record database based on the second-level hash value; if the second-level match fails, performing a first-level match in the historical test record database based on the first-level hash value. It is easy to understand that this is a layered matching method from fine to coarse. This hash-value-based layered matching method can efficiently match the current computational subgraph in the historical test record database. Figure 1 Historical computational subgraphs with similarity or high degree of similarity.

[0062] In some embodiments, the information related to historical test records includes at least one of the following: multi-level feature information of historical computation subgraphs, test result information of historical computation subgraphs, and tested parameter range information of historical computation subgraphs. This ensures the completeness and accuracy of the compilation deduplication evaluation, avoiding evaluation bias and risk omissions caused by missing historical data.

[0063] In some embodiments, the test result information of the historical computation subgraph includes at least one of test status information, test execution time, and test platform information, wherein the test status information indicates whether the test passed or failed. By refining the historical test results into test status information, test execution time, and test platform information, test effectiveness, execution efficiency, and runtime environment can be covered, further improving the comprehensiveness of compilation deduplication evaluation and making the skip test risk quantification results more consistent with actual compilation test scenarios.

[0064] In some embodiments, the compilation deduplication evaluation of the computational subgraph includes evaluating the computational subgraph from at least one of the following dimensions: topology difference dimension, parameter difference dimension, combined difference dimension, and test risk dimension. In some embodiments, the computational subgraph can be evaluated from one of these dimensions. In other embodiments, it can be evaluated from two of these dimensions. In still other embodiments, it can be evaluated from three of these dimensions. In yet another embodiment, it can be evaluated from all four dimensions: topology difference dimension, parameter difference dimension, combined difference dimension, and test risk dimension. By evaluating the computational subgraph from multiple dimensions, the shortcomings of traditional single-dimensional evaluation (one-sidedness and incomplete risk coverage) are overcome. This allows the evaluation results to accurately reflect the testing necessity and potential risks of the current subgraph, providing accurate quantitative basis for subsequent tiered testing decisions.

[0065] The topology difference dimension is used to evaluate the consistency of the current computation subgraph with the matched historical computation subgraph in terms of topological structure. In some embodiments, if the current computation subgraph and the matched historical computation subgraph are completely identical in topology, the topology difference dimension value is 0; if there is a difference in the topology between the current computation subgraph and the matched historical computation subgraph, the topology difference dimension value can be taken in the range of 1 to 10 points according to the degree of difference.

[0066] The parameter difference dimension is used to evaluate whether the operator parameter values ​​of the current computation subgraph fall within the range of previously tested parameters. If all parameter values ​​are within the tested range, the parameter difference dimension value is 0; if any parameter value exceeds the historical tested range, the parameter difference dimension value can be set to a range of 1 to 10, depending on the extent of the out-of-range condition.

[0067] The combined difference dimension is used to evaluate whether the combination pattern of operator nodes in the current computation subgraph is appearing for the first time. If the combination pattern has a historical record, the combined difference dimension value is 0. In some embodiments, when the operator combination pattern appears for the first time, the combined difference dimension value is determined based on the degree of difference between the pattern and the closest pattern in the historical pattern library, the structural complexity of the pattern, and a preset risk pattern table, and is taken in the range of 1 to 10. For example, if the newly appearing pattern is highly similar to a known high-risk compilation pattern, a higher score can be directly assigned, such as 8-10 points; if it is a simple new combination of common operators, a lower score can be assigned, such as 1-3 points.

[0068] The test risk dimension is used to assess the test failure rate of the matched historical computational subgraph, quantifying the potential risk of skipping the current test. The lower the historical failure rate, the lower the test risk dimension value, for example, approaching 0; the higher the failure rate, the higher the value, for example, approaching 10.

[0069] In some embodiments, the evaluation result is calculated using the following formula: Evaluation result = w1 Topological difference dimension value + w2 Parameter difference dimension value + w3 Combined difference dimension values ​​+ w4 The test risk dimension values ​​are defined, where w1, w2, w3, and w4 are the weight coefficients for the corresponding dimension values. These weight coefficients are dynamically adjusted based on historical test data in the historical test record database. Because the weight coefficients for each dimension can be dynamically adjusted based on historical test data, it can adapt to the test risk distribution characteristics of different test scenarios and compiler versions, thus meeting diverse testing requirements.

[0070] In some embodiments, test decisions are determined based on the evaluation results as follows: when the evaluation result is greater than or equal to a first decision threshold, the test decision is to execute a full test; when the evaluation result is greater than or equal to a second decision threshold but less than the first decision threshold, the test decision is to execute a downgraded test, such as testing only on a fast simulator; when the evaluation result is less than the second decision threshold, the test decision is to skip the test; wherein the first decision threshold is greater than the second decision threshold. This allows for differentiated testing strategies to be matched to computational subgraphs with different risks and degrees of difference. It solves the problem of the traditional one-size-fits-all, coarse-grained testing approach, retaining full testing for high-risk, differential computational subgraphs to ensure compilation reliability, streamlining the testing process for medium-risk computational subgraphs to improve efficiency, and eliminating redundant testing for low-risk, repetitive computational subgraphs. While ensuring full coverage and low test misses, it maximizes the reduction of ineffective testing overhead.

[0071] In some embodiments, after performing the corresponding test operations on the neural network compiler under test according to the test decision, the testing method for the neural network compiler provided by the present invention further includes updating the historical test record database in the following ways: storing the feature identifier and multi-level features of the current computation subgraph into the historical test record database; or storing the test result information and test execution information of the current computation subgraph into the historical test record database; or updating the tested parameter range information corresponding to the current computation subgraph in the historical test record database. This enables continuous accumulation and iterative optimization of test data, constantly enriching historical matching samples, improving parameter coverage, and optimizing risk judgment benchmarks, making subsequent matching retrieval more accurate and evaluation quantification more reliable, forming a closed-loop iterative mechanism for test optimization.

[0072] In some embodiments, after updating the historical test record database, the testing method for the neural network compiler provided by this invention further includes: if the test status information of the current computational subgraph indicates that the test has failed, then adjusting the model used to calculate the test risk dimension value according to the failure mode of this test. In this way, the test risk dimension calculation model is dynamically adjusted based on the failure mode for computational subgraphs that have failed the test, solving the problem that traditional risk models cannot adapt to new failure modes and that risk prediction is lagging. This can continuously optimize the accuracy of test risk quantification, improve the ability to predict skipped tests, effectively avoid the risk of missing similar compilation defects, and improve the robustness of compiler testing.

[0073] In some embodiments, the feature identifier of the computational subgraph includes a graph embedding vector; the graph embedding vector is a fixed-dimensional vector generated by encoding the computational subgraph through a pre-trained graph neural network model, wherein the graph embedding vectors corresponding to semantically similar computational subgraphs are close in distance in the vector space.

[0074] In some embodiments, the historical test record database adopts a hierarchical storage architecture: the first layer is a hash index layer, used to store the association mapping relationship between feature identifiers and historical computation subgraphs; the second layer is a vector storage layer, used to store the graph embedding vectors corresponding to the historical computation subgraphs; and the third layer is a structured data layer, used to store multi-level feature information, test result information, and tested parameter range information of the historical computation subgraphs. By dividing the historical test database into a hash index layer, a vector storage layer, and a structured data layer, hierarchical independent storage and classified management of index associations, vector data, and structured test data can be achieved. This optimizes the efficiency of database retrieval, retrieval, and update, enables parallel and efficient processing of fast index matching, similar vector comparison, and structured data query, adapts to multi-level and multi-dimensional subgraph matching and evaluation needs, and improves the overall efficiency of the test optimization process.

[0075] In some embodiments, the technical solution of the present invention can also process the computational subgraphs of multiple neural network models in parallel, executing the computational subgraph extraction operation and multi-level feature calculation operation corresponding to each neural network model in parallel; configure a connection pool structure for query operations on the historical test record database to support high-concurrency access capabilities; and cache the test decision results of the generated computational subgraphs to avoid repeatedly executing feature calculation and matching retrieval processes on the same computational subgraph. This effectively improves the parallel processing efficiency of batch testing of multiple neural network models and reduces redundant computation overhead, making it particularly suitable for large-scale, high-concurrency neural network compiler testing scenarios.

[0076] In some embodiments, the testing method for the neural network compiler provided by the present invention further includes: configuring a multi-level exception handling mechanism; if an exception occurs where the query of the historical test record database fails, the complete test process is executed on the current computational subgraph; if an exception occurs in the compilation deduplication evaluation calculation, a simple matching method based on hierarchical hashing is used to complete the computational subgraph similarity determination; and periodic data backup operations are performed on the historical test record database to prevent the loss of historical test data. This effectively addresses various abnormal conditions during the testing process, avoids test omissions and data loss caused by abnormal conditions, and ensures the stable and reliable operation of the overall testing process.

[0077] In some embodiments, the testing method for the neural network compiler provided by the present invention further includes: real-time monitoring of test metrics such as test deduplication rate, test skip rate, and false positive rate; periodically reviewing abnormal cases that were skipped but subsequently detected as compilation defects, and iteratively optimizing the novelty scoring calculation model and risk assessment logic based on the failure modes of these abnormal cases; conducting comparative tests on different decision thresholds and weight coefficient configurations, comparing the actual operating effects of various parameter configurations, and selecting the optimal parameter configuration suitable for various test scenarios. This enables dynamic iterative optimization of the testing strategy, continuously reducing the probability of false positives, improving the accuracy of deduplication of the computational subgraph, and enhancing the scenario adaptability of the testing scheme of the present invention.

[0078] The following example, using a specific test scenario, will illustrate the specific principles of testing a neural network compiler using the testing method provided by this invention.

[0079] The BERT-Base model (a basic version of the bidirectional encoder representation model) and the BERT-Large model (a large version of the bidirectional encoder representation model) were selected as test cases for the neural network compiler under test. The computational subgraphs of the Transformer layer of these two models were tested and optimized.

[0080] First, a computation graph extraction operation is performed to parse the HLO representations of the BERT-Base model and the BERT-Large model respectively, and extract the complete original computation graph. Both types of models contain multi-layer Transformer structures, and each Transformer structure contains two types of semantically complete computation subgraphs: a self-attention computation subgraph and a feedforward network computation subgraph.

[0081] Then, a computational subgraph decomposition operation is performed, dividing the single-layer Transformer structure into independent test units, with the self-attention computational subgraph as the specific test object. This computational subgraph contains a complete combination of operator nodes. This combination of operator nodes includes, but is not limited to, matrix multiplication operators, batch matrix multiplication operators, random dropout regularization operators, output projection matrix multiplication operators, residual connection operators, and layer normalization operators used for projection computation. These operators constitute semantically complete computational logic units.

[0082] Subsequently, multi-level feature extraction and hierarchical hash calculation operations are performed on the computational subgraph to generate a three-level hash feature identifier for the first self-attention computational subgraph of the BERT-Base model. The first-level hash value is generated based on topological features, using a fixed sequence of operators to generate a topological hash value that fully represents the topological structure of the computational subgraph. The second-level hash value is generated by fusing topological and operator features, overlaying parameters such as the input shape and weight size of each operator node to refine the differences in operator-level features. The third-level hash value is generated by fusing topological, operator, and data flow features, overlaying complete data flow information such as tensor data type and memory layout to accurately represent the full-dimensional semantic features of the computational subgraph. The same computational logic is used to process the first self-attention computational subgraph of the BERT-Large model to generate the corresponding three-level hash value.

[0083] The similarity of the computational subgraphs is determined by multi-level hash matching. For example, if the first-level hash values ​​of the self-attention computational subgraphs of the two models are completely identical, it indicates that their topological structures are completely the same. The second-level hash values ​​are different because the hidden layer dimension of the BERT-Base model is 768, while that of the BERT-Large model is 1024, and the operator parameters are different. The third-level hash values ​​are also different, and the overall full-dimensional features are not completely consistent.

[0084] The compilation deduplication novelty score was completed based on a multi-dimensional difference dimension, quantifying scores from the dimensions of topological novelty, parameter novelty, combinatorial novelty, and testing risk. Topological novelty is scored as 0 points, indicating no difference in the topological structure; parameter novelty is scored as 6 points, indicating that the operator parameters have a reasonable range of differences; combinatorial novelty is scored as 0 points, indicating that the operator combination pattern is a known and mature pattern; and testing risk is scored as 2 points, indicating that the self-attention computation subgraph has a low historical test failure rate and low skip testing risk. The comprehensive evaluation result was calculated using a dynamic weight formula, with weight coefficients w1=0.3, w2=0.5, w3=0.1, and w4=0.1, resulting in a final comprehensive evaluation score of 3.2.

[0085] Subsequently, a test decision is determined based on preset decision thresholds. Assuming that the first decision threshold T1=7 and the second decision threshold T2=3 are preset, since the comprehensive evaluation result of 3.2 points is greater than the second decision threshold and less than the first decision threshold, it can be determined that the current calculation subgraph has no high compilation risk. Finally, the test decision is to downgrade the test, skip the complete compilation test process, and only perform a fast compilation verification through a fast simulator to confirm that the compilation process can pass normally.

[0086] After the test is completed, the historical test record database is updated. The multi-level features and three-level hash features of the BERT-Large model self-attention computation subgraph are stored in the database. The parameter scaling variant association between the BERT-Base model and the computation subgraph of the same origin is established. The range of tested parameters of the corresponding computation subgraph in the database is updated, and the hidden layer dimension coverage range is updated to 768 to 1024.

[0087] The practical application results of the above embodiments show that the technical solution of this application can remove the duplicate parts of the self-attention calculation subgraph and the feedforward network calculation subgraph in the BERT-Large model, reduce the testing time, and thus significantly reduce redundant testing overhead while ensuring the reliability of the test.

[0088] The present invention also provides a testing system 100 for a neural network compiler, such as... Figure 3 As shown, the test system 100 for the neural network compiler includes: Test case acquisition module 101 is configured to acquire the neural network model as the test case of the neural network compiler to be tested; The computation graph generation and preprocessing module 102 is configured to generate the original computation graph of the neural network model and perform semantic normalization preprocessing on the original computation graph to obtain the intermediate computation graph, wherein the semantically equivalent intermediate computation graphs have the same structured feature sequence. The computation graph decomposition module 103 is configured to decompose an intermediate computation graph into multiple computation subgraphs, wherein each computation subgraph is a semantically complete computation subgraph. The feature extraction and label generation module 104 is configured to extract multi-level features of each computational subgraph and generate feature labels of the computational subgraph based on the multi-level features, wherein the multi-level features include topological features, operator features and data flow features. The matching retrieval module 105 is configured to perform multi-level matching retrieval in the historical test record database based on feature identifiers. The historical test record database stores feature identifiers of historical calculation subgraphs and related information of historical test records associated with the feature identifiers of historical calculation subgraphs. The deduplication evaluation module 106 is configured to compile and deduplicate the computational subgraph that retrieves matching historical test record information from the historical test record database, and obtain the evaluation result. The evaluation result represents the degree of similarity between the computational subgraph and the matched historical computational subgraph and the degree of risk of skipping the test. The decision-making module 107 is configured to determine test decisions based on the evaluation results, wherein the test decisions include one of executing a full test, executing a downgraded test, and skipping the test; The decision execution module 108 is configured to perform corresponding test operations on the compiler of the neural network to be tested according to the test decision, so as to obtain test results. The test operations include hierarchical compilation and verification of the neural network model.

[0089] Regarding the test system 100 for the neural network compiler in the above embodiments, the specific manner in which the processor executes operations has been described in detail in the embodiments related to the method, and will not be elaborated upon here. The method embodiments and system embodiments of the present invention can complement each other.

[0090] In summary, the technical solution provided by this invention has the following advantages: The technical solution preprocesses the original computation graph through semantic normalization to obtain an intermediate computation graph. Semantically equivalent intermediate computation graphs share the same structured feature sequence, providing a unified representation basis for computation graph comparison. Based on this, feature identifiers for computation subgraphs are generated. Since these feature identifiers are generated based on the extracted multi-level features of each computation subgraph, they carry features from multiple levels of the computation subgraph. Therefore, multi-level matching retrieval is performed in the historical test record database based on these feature identifiers, enabling efficient and accurate location of historical test records similar to the current computation subgraph. Simultaneously, by compiling and deduplicating the computation subgraphs that have been matched with historical records, an evaluation result is obtained. Since the evaluation result characterizes the similarity between the current computation subgraph and the matched historical computation subgraph, as well as the risk of skipping tests, the similarity between the current computation subgraph and historical records, and the risk of skipping tests, can be quantified based on the obtained evaluation result. Based on this, corresponding test decisions are determined based on the evaluation result, and corresponding test operations are executed. This achieves accurate identification and elimination of redundant test cases while maintaining test coverage, improving the efficiency of test resource utilization.

[0091] Embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, wherein when the executable computer program in the storage medium is executed by a processor, it can implement the testing method of the neural network compiler as described in any of the above embodiments.

[0092] Embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements a testing method for a neural network compiler as described in any of the above embodiments.

[0093] Furthermore, embodiments of the present invention also provide an electronic device 300, such as... Figure 4 As shown, the electronic device 300 includes a memory 301 and a processor 302. The memory 301 is used to store computer programs executable by the processor 302; the processor 302 is used to execute the computer programs in the memory 301 to implement the test method of the neural network compiler as described in any of the above embodiments.

[0094] The electronic device 300 also includes a communication interface 303. The processor 302, memory 301, and communication interface 303 are connected via a communication bus and communicate with each other.

[0095] Processor 302 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the above scheme program.

[0096] Communication interface 303 is used to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Networks (WLAN), etc.

[0097] Memory 301 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory may exist independently and be connected to the processor via a bus. Memory may also be integrated with the processor.

[0098] In this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The term "multiple" refers to two or more unless otherwise expressly defined.

[0099] The above description of the embodiments is intended to enable those skilled in the art to understand and apply the present invention. It will be apparent to those skilled in the art that various modifications can be made to these embodiments, and the general principles described herein can be applied to other embodiments without creative effort. Therefore, the present invention is not limited to the embodiments described herein, and any improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope and spirit of the invention are within the scope of the present invention.

Claims

1. A testing method for a neural network compiler, characterized in that, include: Obtain the neural network model as the test case for the compiler of the neural network to be tested; The original computation graph of the neural network model is generated, and the original computation graph is subjected to semantic normalization preprocessing to obtain an intermediate computation graph, wherein the semantically equivalent intermediate computation graphs have the same structured feature sequence. The intermediate computation graph is decomposed into multiple computation subgraphs, wherein each computation subgraph is a semantically complete computation subgraph; Extract multi-level features from each computational subgraph and generate feature identifiers for the computational subgraph based on the multi-level features, wherein the multi-level features include topological features, operator features, and data flow features; Multi-level matching retrieval is performed in the historical test record database based on the feature identifiers, wherein the historical test record database stores feature identifiers of historical calculation subgraphs and related information of historical test records associated with the feature identifiers of the historical calculation subgraphs; For the computational subgraph that retrieves matching historical test record information from the historical test record database, a compilation and deduplication evaluation is performed to obtain an evaluation result, wherein the evaluation result characterizes the similarity between the computational subgraph and the matching historical computational subgraph and the risk of skipping the test; A testing decision is determined based on the evaluation results, wherein the testing decision includes one of executing a full test, executing a downgraded test, or skipping the test; Based on the test decision, corresponding test operations are performed on the compiler of the neural network to be tested to obtain test results. The test operations include hierarchical compilation and verification of the neural network model.

2. The testing method for a neural network compiler according to claim 1, characterized in that, The semantic normalization preprocessing performed on the original computation graph includes at least one of the following operations: Perform a standardized naming operation on all operator nodes in the original computation graph to unify the names of operator nodes with the same function; A pre-computation operation is performed on the operator nodes in the original computation graph that take only constants as input, and the operator nodes that take only constants as input are replaced with the pre-computed result values; The execution result in the original computation graph does not affect the operator nodes and connecting edges in the final output of the original computation graph. as well as Based on the dependencies of the computational logic, the topological order of all nodes in the original computation graph is determined, so that semantically equivalent original computation graphs have a consistent node arrangement order.

3. The testing method for a neural network compiler according to claim 1, characterized in that, The topological features include at least one of the following: the number of nodes in the computational subgraph, the connection relationships between nodes, the depth of the computational subgraph, and the width of the computational subgraph.

4. The testing method for a neural network compiler according to claim 1, characterized in that, The operator features include the operator type and / or parameters of each operator node in the computation subgraph.

5. The testing method for a neural network compiler according to claim 1, characterized in that, The data flow characteristics include at least one of the following: the shape, data type, and memory layout information of the tensors in the computation subgraph.

6. The testing method for a neural network compiler according to claim 1, characterized in that, The feature identifiers of the computational subgraph include first-level hash values, second-level hash values, and third-level hash values; The first-level hash value is generated based on the topological features; the second-level hash value is generated based on the topological features and the operator features; and the third-level hash value is generated based on the topological features, the operator features, and the data flow features.

7. The testing method for a neural network compiler according to claim 6, characterized in that, Also includes: A third-level match is performed in the historical test record database based on the third-level hash value. If the third-level match fails, a second-level match is performed in the historical test record database based on the second-level hash value. If the second-level match fails, a first-level match is performed in the historical test record database based on the first-level hash value.

8. The testing method for a neural network compiler according to claim 1, characterized in that, The information related to the historical test records includes at least one of the following: multi-level feature information of the historical calculation subgraph, test result information of the historical calculation subgraph, and tested parameter range information of the historical calculation subgraph.

9. The testing method for a neural network compiler according to claim 8, characterized in that, The test result information of the historical calculation subgraph includes at least one of test status information, test execution time, and test platform information, wherein the test status information indicates whether the test passed.

10. The testing method for a neural network compiler according to claim 1, characterized in that, The computational subgraph is evaluated by compilation and deduplication, including: The computational subgraph is evaluated for compilation and deduplication based on at least one of the following dimensions: topological difference dimension, parameter difference dimension, combinatorial difference dimension, and test risk dimension.

11. The testing method for a neural network compiler according to claim 10, characterized in that, The evaluation result is calculated using the following formula: Evaluation result = w1 Topological difference dimension value + w2 Parameter difference dimension value + w3 Combined difference dimension values ​​+ w4 The test risk dimension values ​​are defined, where w1, w2, w3, and w4 are the weight coefficients for the corresponding dimension values. The weight coefficients w1, w2, w3, and w4 are dynamically adjusted based on historical test data in the historical test record database.

12. The testing method for a neural network compiler according to claim 11, characterized in that, The testing decision is determined based on the evaluation results in the following manner: When the evaluation result is greater than or equal to the first decision threshold, the test decision is determined to execute the complete test; When the evaluation result is greater than or equal to the second decision threshold and less than the first decision threshold, the test decision is determined to be to perform a degradation test. When the evaluation result is less than the second decision threshold, the test decision is determined to be to skip the test; Wherein, the first decision threshold is greater than the second decision threshold.

13. The testing method for a neural network compiler according to claim 1, characterized in that, After performing the corresponding test operations on the compiler of the neural network under test according to the test decision, the method further includes updating the historical test record database in the following manner: Store the feature identifier and multi-level features of the current computation subgraph into the historical test record database; or store the test result information and test execution information of the current computation subgraph into the historical test record database; or update the tested parameter range information corresponding to the current computation subgraph in the historical test record database.

14. The testing method for a neural network compiler according to claim 11, characterized in that, After updating the historical test record database, the method further includes: If the test status information of the current computation subgraph is "test failed", then the model used to calculate the test risk dimension value is adjusted according to the failure mode of this test.

15. The testing method for a neural network compiler according to claim 1, characterized in that, The historical test record database adopts a hierarchical storage architecture: the first layer is a hash index layer, which is used to store the association mapping relationship between the feature identifier and the historical calculation subgraph; The second layer is a vector storage layer, used to store the graph embedding vectors corresponding to the historical computation subgraphs; the third layer is a structured data layer, used to store the multi-level feature information, test result information, and tested parameter range information of the historical computation subgraphs.

16. A testing system for a neural network compiler, characterized in that, include: The test case acquisition module is configured to acquire the neural network model that serves as the test case for the neural network compiler under test. The computation graph generation and preprocessing module is configured to generate the original computation graph of the neural network model and perform semantic normalization preprocessing on the original computation graph to obtain an intermediate computation graph, wherein the semantically equivalent intermediate computation graphs have the same structured feature sequence. The computation graph decomposition module is configured to decompose the intermediate computation graph into multiple computation subgraphs, wherein each computation subgraph is a semantically complete computation subgraph; The feature extraction and label generation module is configured to extract multi-level features of each computational subgraph and generate feature labels of the computational subgraph based on the multi-level features, wherein the multi-level features include topological features, operator features and data flow features; The matching and retrieval module is configured to perform multi-level matching and retrieval in the historical test record database based on the feature identifier, wherein the historical test record database stores the feature identifier of the historical calculation subgraph and related information of the historical test record associated with the feature identifier of the historical calculation subgraph. The deduplication evaluation module is configured to compile and deduplicate the computational subgraph that retrieves matching historical test record information from the historical test record database to obtain an evaluation result, wherein the evaluation result characterizes the similarity between the computational subgraph and the matched historical computational subgraph and the risk of skipping the test; The decision-making module is configured to determine a test decision based on the evaluation results, wherein the test decision includes one of executing a full test, executing a downgraded test, and skipping the test; The decision execution module is configured to perform corresponding test operations on the neural network compiler under test according to the test decision, so as to obtain test results. The test operations include hierarchical compilation and verification of the neural network model.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the executable computer program in the storage medium is executed by a processor, it enables the testing method of the neural network compiler as described in any one of claims 1 to 15.

18. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the test method of the neural network compiler as described in any one of claims 1 to 15.

19. An electronic device, characterized in that, The system includes a memory and a processor, wherein the memory stores a computer program executable by the processor; and the processor executes the computer program in the memory to implement a test method for a neural network compiler as described in any one of claims 1 to 15.

Images