Deep learning framework test method and device, computer equipment and storage medium

Abstract

The embodiment of the invention discloses a deep learning framework test method and device, computer equipment and a storage medium. The method comprises the following steps: creating a seed tensor, and performing mutation to obtain a test input tensor; creating a seed model, and performing mutation to obtain a test input model; under different browsers, performing differential testing on a to-be-tested deep learning framework according to the test input tensor and the test input model to obtain a test result; and adjusting a mutation rule corresponding to the seed tensor and the seed model according to the test result. By implementing the method provided by the embodiment of the invention, a JavaScript deep learning framework focused on an edge environment can be realized, and framework defects related to a reasoning optimization mechanism can be more comprehensively detected, so that the technical bottleneck existing in the JavaScript environment in an existing deep learning framework test method is solved.

Description

Technical Field

[0001] The present invention relates to the field of deep learning, and more specifically to a deep learning framework testing method, apparatus, computer equipment and storage medium. Background Art

[0002] With the popularity of web applications, JavaScript has become the main language for front-end development. This trend has led to an increasing demand for integrating deep learning models into web applications. However, this also brings challenges to testing deep learning models, especially in terms of input tensor generation and model generation.

[0003] In testing input tensor generation, current testing methods for deep learning frameworks often rely on fixed tensor shapes and data types, which limits comprehensive testing of cache reuse mechanisms. For example, existing methods may not be able to effectively detect modules that process different tensor shapes. In addition, JavaScript deep learning frameworks may misjudge whether two tensors are exactly the same, that is, two tensors with the same shape but different data are mistakenly considered to be the same. This misjudgment leads to cache reuse errors, but existing testing methods fail to effectively identify this problem.

[0004] In terms of test input model generation, existing deep learning framework testing methods often fail to generate models that can trigger inference acceleration mechanisms. These methods mainly target mainstream frameworks such as TensorFlow and PyTorch, but fail to consider deep learning frameworks in JavaScript environments. Therefore, these testing methods are difficult to detect framework defects caused by inference acceleration mechanisms (such as node optimization, operator reordering, and operator fusion).

[0005] Therefore, it is necessary to design a new method to focus on the JavaScript deep learning framework in the edge environment and more comprehensively detect framework defects related to the inference optimization mechanism to solve the technical bottlenecks of the existing deep learning framework testing methods in the JavaScript environment. Summary of the invention

[0006] The purpose of the present invention is to overcome the defects of the prior art and provide a deep learning framework testing method, device, computer equipment and storage medium.

[0007] To achieve the above purpose, the present invention adopts the following technical solution: a deep learning framework testing method, comprising:

[0008] Create a seed tensor and mutate it to get the test input tensor;

[0009] Create a seed model and mutate it to get the test input model;

[0010] Under different browsers, differential testing is performed on the deep learning framework to be tested according to the test input tensor and the test input model to obtain a test result;

[0011] The mutation rules corresponding to the seed tensor and the seed model are adjusted according to the test results.

[0012] A further technical solution is: the step of creating a seed tensor and mutating it to obtain a test input tensor includes:

[0013] Create a seed tensor in the format of (N, C, H, W), where N is the number of batches, C is the number of channels, H is the height, and W is the width;

[0014] The seed tensor is mutated using a mutation rule of the tensor to obtain a test input tensor, wherein the mutation rule of the tensor includes at least one of tensor copying, padding, transposing, clipping, and data type conversion.

[0015] A further technical solution is: the step of creating a seed model and performing mutation to obtain a test input model includes:

[0016] Creating a seed model, wherein the structure of the seed model is an acyclic chain consisting of identity operators;

[0017] The seed model is mutated using a mutation rule of the model to obtain a test input model, wherein the mutation rule includes at least one of zero-dimensional tensorization, non-shift transposition, dimensionality reduction averaging of height-width dimensions, transposition-based operator rearrangement, maximum pooling-based operator rearrangement, equivalent separable convolution, equivalent convolution-normalization-activation block, and random operator.

[0018] A further technical solution is: in different browsers, differential testing is performed on the deep learning framework to be tested according to the test input tensor and the test input model to obtain a test result, including:

[0019] Under different browsers, the test input tensor and the test input model are used as input, and the deep learning framework to be tested is used as a test object, and differential testing is performed to obtain test results.

[0020] A further technical solution is: adjusting the mutation rules corresponding to the seed tensor and the seed model according to the test results includes:

[0021] Calculate the effect index corresponding to the test result, and guide the selection of mutation rules corresponding to the test input tensor and the test input model;

[0022] Record the contribution of each mutation rule to the test results of the deep learning framework to be tested, and adjust the selection probability of the mutation rule.

[0023] A further technical solution is: calculating the effect index corresponding to the test result and guiding the selection of mutation rules corresponding to the test input tensor and the test input model, including:

[0024] When the detection framework to be tested crashes or there is a non-numerical defect in the detection framework to be tested, the test result indicates that the test input model fails to trigger the corresponding defect, and the effect indicator corresponding to the test result is set to 0; the test result indicates that the test input model successfully triggers the corresponding defect, and the effect indicator corresponding to the test result is set to the average value of the test input tensor value; when the test result indicates a detection inconsistency defect, the effect indicator corresponding to the test result is set to the maximum value of the output tensor difference of the test results under different browsers;

[0025] The selection of mutation rules corresponding to the test input tensor and the test input model is guided by the effect index.

[0026] A further technical solution is: recording the contribution of each mutation rule to the test result of the deep learning framework to be tested and adjusting the selection probability of the mutation rule, including:

[0027] The contribution of each mutation rule to the test result of the deep learning framework to be tested is calculated using c1=c0+Δfitness, where c1 and c0 are the overall contributions of the mutation rule to the test result before and after the mutation, and Δfitness=fitness new -fitness old Among them, fitness new Represents the effect index of the newly generated test input model after mutation; fitness old Represents the effect index of the test input model before mutation;

[0028] According to the contribution Control the probability of the mutation rule being selected again, where n represents the total number of mutation rules and p represents the probability of the mutation rule being selected again.

[0029] The present invention also provides a deep learning framework testing device, comprising:

[0030] A test input tensor generation unit, which is used to create a seed tensor and mutate it to obtain a test input tensor;

[0031] A test input model generation unit, used to create a seed model and perform mutation to obtain a test input model;

[0032] A defect detection unit, used for performing differential testing on the deep learning framework to be tested according to the test input tensor and the test input model under different browsers to obtain a test result;

[0033] The heuristic guidance unit is used to adjust the mutation rules corresponding to the seed tensor and the seed model according to the test results.

[0034] The present invention further provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor implements the above method when executing the computer program.

[0035] The present invention also provides a storage medium, wherein the storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0036] The beneficial effects of the present invention compared with the prior art are: the present invention creates seed tensors and models, applies mutation rules to generate test inputs and models; performs differential testing on deep learning frameworks in different environments to evaluate the effects; adjusts mutation rules based on test results to optimize the testing process; and implements a JavaScript deep learning framework focused on edge environments, more comprehensively detecting framework defects related to inference optimization mechanisms, so as to solve the technical bottlenecks of existing deep learning framework testing methods in JavaScript environments.

[0037] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0039] Figure 1 A schematic diagram of an application scenario of the deep learning framework testing method provided by an embodiment of the present invention;

[0040] Figure 2 A schematic diagram of a deep learning framework testing method provided by an embodiment of the present invention;

[0041] Figure 3 A schematic diagram of a sub-process of a deep learning framework testing method provided by an embodiment of the present invention;

[0042] Figure 4 A schematic diagram of a sub-process of a deep learning framework testing method provided by an embodiment of the present invention;

[0043] Figure 5 A schematic diagram of a sub-process of a deep learning framework testing method provided by an embodiment of the present invention;

[0044] Figure 6 A schematic diagram of a sub-process of a deep learning framework testing method provided by an embodiment of the present invention;

[0045] Figure 7 A schematic diagram of a sub-process of a deep learning framework testing method provided by an embodiment of the present invention;

[0046] Figure 8 A schematic block diagram of a deep learning framework testing device provided by an embodiment of the present invention;

[0047] Fig. 9 A schematic block diagram of a test input tensor generation unit of a deep learning framework testing device provided by an embodiment of the present invention;

[0048] Fig.10 A schematic block diagram of a test input model generation unit of a deep learning framework testing device provided by an embodiment of the invention;

[0049] Fig.11 A schematic block diagram of a heuristic guidance unit of a deep learning framework testing device provided by an embodiment of the invention;

[0050] Fig.12 A schematic block diagram of an indicator calculation subunit of a deep learning framework testing device provided by an embodiment of the invention;

[0051] Fig.13 A schematic block diagram of an adjustment subunit of a deep learning framework testing device provided by an embodiment of the present invention;

[0052] Fig.14 A schematic block diagram of a computer device provided by an embodiment of the invention. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0055] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0056] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0057] See also Figure 1 and Figure 2 , Figure 1 A schematic diagram of an application scenario of a deep learning framework testing method provided in an embodiment of the present invention, specifically a method for a deep learning framework in a browser-side JavaScript environment, Figure 2 A schematic flow chart of a deep learning framework testing method provided for an embodiment of the present invention. The deep learning framework testing method is applied to a terminal, on which multiple browsers are integrated, and test input tensors of various shapes and data types are generated through mutations, and a test input model is formed through mutations, and the test input model focuses on creating a model that can trigger three inference optimization strategies, including node optimization, operator fusion, and operator rearrangement. The generated test input tensors are used to perform differential testing in different browsers, and different test predictions are applied to the defects in the deep learning detection framework to be tested, including crashes, non-number defects, and inconsistent defects. Heuristic indicators are then calculated based on the detection effects of the test input tensors and models generated in each round, thereby guiding the selection of mutation rules for the test input tensors and models in subsequent rounds to continuously improve the detection effect of framework defects.

[0058] Figure 2 Schematic diagram of the process of the deep learning framework testing method provided by the embodiment of the present invention. Figure 2 As shown, the method includes the following steps S110 to S140.

[0059] S110. Create a seed tensor and mutate it to obtain a test input tensor.

[0060] In this embodiment, the test input tensor refers to the input data tensor used to evaluate the performance of the model after the model is trained. A tensor is a multidimensional array of data and can be regarded as a multidimensional matrix.

[0061] In one embodiment, see Figure 3 , the above-mentioned step S110 may include steps S111 to S112.

[0062] S111. Create a seed tensor in the (N, C, H, W) format, where N is the number of batches, C is the number of channels, H is the height, and W is the width.

[0063] In this example, a tensor with a specific shape is created, the shape is (N, C, H, W).

[0064] N: batch size, which indicates the number of samples input into the model at one time.

[0065] C: Number of channels. For images, this is usually 3 (RGB images) or 1 (grayscale images).

[0066] H: Height, the number of vertical pixels of the image.

[0067] W: Width, the horizontal number of pixels of the image.

[0068] Make sure the shape of the test data is consistent with the expected input of the model so that the model can process the data correctly. You can test the impact of data of different sizes and dimensions on model performance by adjusting the values ​​of N, C, H, and W.

[0069] S112. Mutate the seed tensor using a tensor mutation rule to obtain a test input tensor, wherein the tensor mutation rule includes at least one of tensor copying, padding, transposing, cropping, and data type conversion.

[0070] In this embodiment, the above mutation rules include:

[0071] Tensor replication: The seed tensor is replicated multiple times to generate new test input tensors. This can increase the sample size and improve the robustness of the test.

[0072] Padding: Add extra pixels to the borders of a tensor. Useful for testing how a model performs with different input sizes.

[0073] Transpose: Change the order of the dimensions of a tensor. Used to test the performance of the model under different data arrangements.

[0074] Cropping: Extracting sub-regions from a tensor. This allows testing the performance of the model on partial input data.

[0075] Data type conversion: Change the data type of a tensor (e.g., from floating point to integer). This helps test how well the model handles different data formats.

[0076] The details are shown in Table 1.

[0077] Table 1. Mutation rules for test input tensors

[0078]

[0079] In view of the fact that the cache reuse mechanism in the JavaScript environment is sensitive to changes in tensors and data types, the shape and data type of the test input tensor are changed through mutation. Through different mutation rules, diversified test inputs can be created to cover more scenarios and conditions. The performance of the model under different input variants can be detected to help identify the model's weaknesses and room for improvement. The model's sensitivity to different data processing methods can be tested, thereby improving the stability and reliability of the model in practical applications.

[0080] Combined, these steps ensure that the test input tensors are effective in evaluating the performance of the model under a variety of conditions, helping to optimize and validate the model.

[0081] S120: Create a seed model and mutate it to obtain a test input model.

[0082] In this embodiment, the test input model refers to the data model used for testing, that is, the input data used to evaluate and verify the trained deep learning framework to be tested. It ensures that the performance of the deep learning framework to be tested on new and unseen data is checked, so as to judge the generalization ability of the deep learning framework to be tested. In short, the accuracy and effect of the deep learning framework to be tested are verified by using the test data.

[0083] Start by creating a seed model and then apply mutation rules to generate test input models. This helps evaluate and optimize different variants of the model structure to improve overall performance and robustness.

[0084] In one embodiment, see Figure 4 , the above-mentioned step S120 may include steps S121 to S122.

[0085] S121. Create a seed model, wherein the structure of the seed model is an acyclic chain composed of identity operators.

[0086] In this embodiment, the seed model is a Directed Acyclic Graph (DAG) composed of identity operators. In this model, each operator directly passes input data to output without performing any actual data transformation or calculation.

[0087] Models of this structure are usually simple and serve as the basis for test or benchmark models to test the impact of structural changes on the final performance.

[0088] The seed model has a simple structure and is easy to understand and analyze. Using a simple seed model can serve as a standard basis for testing other complex variants. Model complexity can be avoided during the testing phase in order to focus on the impact of mutation rules on model performance.

[0089] S122. Mutate the seed model using the mutation rule of the model to obtain a test input model, wherein the mutation rule includes at least one of zero-dimensional tensorization, non-shift transposition, dimensionality reduction and averaging of height-width dimensions, transposition-based operator rearrangement, maximum pooling-based operator rearrangement, equivalent separable convolution, equivalent convolution-normalization-activation block, and random operator.

[0090] In this embodiment, the mutation rules and their definitions include:

[0091] Zero-dimensional tensorization: Reducing the dimension of the tensor in the model to zero dimension is usually used to test the model's ability to handle extreme situations.

[0092] Transpose without shifting: Modify the transpose operation of the model without changing the data position. Mainly used to test the performance of the model under different data arrangements.

[0093] Height-width dimension reduction and averaging: Reduce the height and width dimensions in the model, usually by calculating the average. This helps test the robustness of the model to size changes.

[0094] Operator rearrangement based on transposition: Operators in the model are transposed to test their performance in different orderings.

[0095] Operator rearrangement based on max pooling: Rearrange the operators in the model using the max pooling operation to evaluate the performance of the model when processing the pooling layer.

[0096] Equivalent Separable Convolution: Replace the standard convolution operation with a separable convolution to reduce computational complexity and test the model’s adaptability to this transformation.

[0097] Equivalent Convolution-Normalization-Activation Block: The convolution layer is combined with the normalization layer and the activation layer into an equivalent block to simplify the model and test its adaptability to this structural optimization.

[0098] At least one of the random operators: Introduce randomness into the model, such as randomly selecting operators or parameters, to test the robustness of the model to random changes.

[0099] The details are shown in Table 2.

[0100] Table 2. Mutation rules for testing input models

[0101]

[0102]

[0103] Based on the inference acceleration mechanism in the JavaScript deep learning framework, a deep learning model that can trigger more inference acceleration strategies is generated. The inference acceleration strategies targeted by the model mutation rules include three categories: node optimization, operator rearrangement, and operator fusion; Node optimization: Improve each node in the calculation graph, such as by replacing complex operations with simpler operations or reducing redundant calculations to improve computing efficiency. Operator rearrangement: Adjust the order of operators in the calculation graph to more effectively utilize hardware resources or improve data access efficiency, thereby accelerating the inference process. Operator fusion: Combine multiple operators into a composite operator to reduce intermediate data storage and transmission overhead, which can improve overall computing efficiency and reduce inference time.

[0104] By applying different mutation rules, a wider range of test scenarios and conditions can be covered; by transforming the model structure, a more efficient model architecture can be found to improve the model's computational efficiency and prediction accuracy; testing the model's sensitivity to different transformations helps identify and improve the model's potential weaknesses; by introducing a variety of mutation rules, the model's performance under different conditions can be tested to ensure its stability in practical applications.

[0105] Through these steps, different variations of the model can be systematically created and evaluated, optimizing the model structure and thus improving the overall performance and adaptability.

[0106] S130. Under different browsers, perform differential testing on the deep learning framework to be tested according to the test input tensor and the test input model to obtain a test result.

[0107] In this embodiment, under different browsers, the test input tensor and the test input model are used as input, and the deep learning framework to be tested is used as a test object, and differential testing is performed to obtain test results.

[0108] Differential testing is performed between different browsers. By comparing the differences between the results obtained by processing the same test input under different browsers, defects in the JavaScript deep learning framework can be detected.

[0109] Specifically, the purpose of differential testing is to detect potential defects in the framework by running the same deep learning framework and the same test input (including test input models and tensors) under different browsers. This approach helps identify and fix inconsistencies and computational errors in models in different environments.

[0110] First, generate the same test input model and tensors. These inputs will be used in all browsers to ensure the consistency of the test; apply the same test input in different browser environments and record the output results in each environment; identify and locate inconsistency defects in the framework by comparing the differences between the output result tensors under different browsers; check whether non-number (NaN) values ​​appear in the output result tensors to find out calculation errors or numerical stability problems in the framework.

[0111] By comparing the results under different browsers, we can discover and resolve inconsistencies in the framework in different environments, thereby improving the stability and consistency of the model; this testing method can effectively identify defects in the framework, including calculation errors and numerical instability, thereby reducing unexpected behaviors in actual applications; differential testing helps ensure that the deep learning framework performs consistently in different browsers or environments, thereby improving its cross-platform compatibility and reliability; detecting and fixing non-numeric problems helps improve calculation accuracy and ensure the accuracy and reliability of model prediction results.

[0112] S140. Adjust the mutation rules corresponding to the seed tensor and the seed model according to the test results.

[0113] In one embodiment, see Figure 5 , the above-mentioned step S140 may include steps S141 to S142.

[0114] S141. Calculate the effect index corresponding to the test result, and guide the selection of mutation rules corresponding to the test input tensor and the test input model.

[0115] In one embodiment, see Figure 6 , the above-mentioned step S140 may include steps S1411 to S1412.

[0116] S1411. When the detection framework to be tested crashes or there is a non-numerical defect in the detection framework to be tested, the test result indicates that the test input model fails to trigger the corresponding defect, and the effect indicator corresponding to the test result is set to 0; the test result indicates that the test input model successfully triggers the corresponding defect, and the effect indicator corresponding to the test result is set to the average value of the test input tensor value; when the test result indicates an inconsistent detection defect, the effect indicator corresponding to the test result is set to the maximum value of the output tensor difference of the test results under different browsers.

[0117] In this embodiment, in the test of the generated model, the mutation rules of the subsequent test input tensors and models are guided by calculating the heuristic indicators, thereby enhancing the framework effect of the test method. Specifically, the indicator fitness is used to describe the effect of the model detection framework defects. For different defect types, the calculation method of fitness is as follows:

[0118] Framework crash and non-number defect: If the model fails to trigger a framework crash or non-number defect, fitness is 0; if it is successfully triggered, fitness is the average value of the test input tensor.

[0119] Inconsistency defect: fitness is the maximum value of the output result tensor difference under different browsers.

[0120] By dynamically adjusting mutation rules, the effectiveness and accuracy of testing can be improved; appropriate calculation methods can be used for different types of defects to more comprehensively detect potential problems in the framework; heuristic indicators are used to guide the selection of test inputs to help discover and fix defects faster.

[0121] S1412. Guide the selection of mutation rules corresponding to the test input tensor and the test input model according to the effect index.

[0122] In this embodiment, the test input tensor and the mutation rule of the model are selected according to the effect index, so as to optimize the test process.

[0123] This approach ensures that the mutation rules selected during the test process can maximize the exposure of potential defects, thereby improving the effectiveness and efficiency of the test. By focusing on performance indicators, the test can identify and solve problems more specifically, reducing unnecessary testing and waste of resources.

[0124] S142. Record the contribution of each mutation rule to the test result of the deep learning framework to be tested, and adjust the selection probability of the mutation rule.

[0125] In one embodiment, see Figure 7 , the above-mentioned step S142 may include steps S1421 to S1422.

[0126] S1421. Use c1=c0+Δfitness to calculate the contribution of each mutation rule to the test result of the deep learning framework to be tested, where c1 and c0 are the overall contribution of the mutation rule to the test result before and after the mutation, and Δfitness=fitness new -fitness old Among them, fitness new Represents the effect index of the newly generated test input model after mutation; fitness old Represents the effect index of the test input model before mutation.

[0127] In this way, the impact of each mutation rule on the test performance can be quantified.

[0128] S1422. Adopting the contribution Control the probability of the mutation rule being selected again, where n represents the total number of mutation rules and p represents the probability of the mutation rule being selected again.

[0129] By calculating the actual contribution of mutation rules through Δfitness, the impact of each rule on the test effect can be accurately evaluated. This helps to understand which mutation rules can more effectively reveal defects, thereby better guiding the test strategy. The contribution of mutation rules is used to adjust their selection probability, so that rules with high contributions have a higher probability of being selected. This method can make the test more focused on mutation rules that show good results, thereby improving test efficiency and coverage. By quantifying and adjusting the contribution of mutation rules, the test input model can be continuously optimized, more potential defects can be discovered, and the detection capability of the test framework can be improved. This method can dynamically adjust the test strategy according to the changes in real-time test results, making the test process more adaptive and flexible, and can continuously optimize and adjust the test rules at different test stages.

[0130] The method of this embodiment generates test input tensors for the cache reuse mechanism in the JavaScript environment, and generates a test input model for the inference acceleration mechanism in the JavaScript environment to detect defects in the JavaScript deep learning framework. It is highly practical for developers and testers of the deep learning framework.

[0131] The above-mentioned deep learning framework testing method creates seed tensors and models, applies mutation rules to generate test inputs and models; performs differential testing on the deep learning framework in different environments to evaluate the effect; adjusts the mutation rules according to the test results to optimize the testing process; and implements a JavaScript deep learning framework focused on the edge environment to more comprehensively detect framework defects related to the inference optimization mechanism, so as to solve the technical bottlenecks of existing deep learning framework testing methods in the JavaScript environment.

[0132] Figure 8 is a schematic block diagram of a deep learning framework testing device 300 provided in an embodiment of the present invention. Figure 8 As shown, corresponding to the above deep learning framework testing method, the present invention also provides a deep learning framework testing device 300. The deep learning framework testing device 300 includes a unit for executing the above deep learning framework testing method, and the device can be configured in a terminal. Specifically, please refer to Figure 8 The deep learning framework testing device 300 includes a test input tensor generation unit 301, a test input model generation unit 302, a defect detection unit 303 and a heuristic guidance unit 304.

[0133] The test input tensor generation unit 301 is used to create a seed tensor and mutate it to obtain a test input tensor; the test input model generation unit 302 is used to create a seed model and mutate it to obtain a test input model; the defect detection unit 303 is used to perform differential testing on the deep learning framework to be tested according to the test input tensor and the test input model under different browsers to obtain test results; the heuristic guidance unit 304 is used to adjust the mutation rules corresponding to the seed tensor and the seed model according to the test results.

[0134] In one embodiment, if Fig. 9 As shown, the test input tensor generation unit 301 includes a seed tensor creation subunit 3011 and a tensor mutation subunit 3012 .

[0135] The seed tensor creation subunit 3011 is used to create a seed tensor in the (N, C, H, W) format, where N is the number of batches, C is the number of channels, H is the height, and W is the width; the tensor mutation subunit 3012 is used to mutate the seed tensor using the tensor mutation rule to obtain a test input tensor, where the tensor mutation rule includes at least one of tensor copying, padding, transposing, cropping, and data type conversion.

[0136] In one embodiment, if Fig.10 As shown, the test input model generating unit 302 includes a seed model creating subunit 3021 and a model mutation subunit 3022 .

[0137] A seed model creation subunit 3021 is used to create a seed model, wherein the structure of the seed model is an acyclic chain composed of identity operators; a model mutation subunit 3022 is used to mutate the seed model using the mutation rule of the model to obtain a test input model, wherein the mutation rule includes at least one of zero-dimensional tensorization, non-shift transposition, dimensionality reduction averaging of height-width dimensions, transposition-based operator rearrangement, maximum pooling-based operator rearrangement, equivalent separable convolution, equivalent convolution-normalization-activation block, and random operator.

[0138] In one embodiment, the defect detection unit 303 is used to perform differential testing under different browsers, using the test input tensor and the test input model as input and the deep learning framework to be tested as a test object to obtain test results.

[0139] In one embodiment, if Fig.11 As shown, the heuristic guidance unit 304 includes an indicator calculation subunit 3041 and an adjustment subunit 3042 .

[0140] The indicator calculation subunit 3041 is used to calculate the effect indicator corresponding to the test result and guide the selection of mutation rules corresponding to the test input tensor and the test input model; the adjustment subunit 3042 is used to record the contribution of each mutation rule to the test result of the deep learning framework to be tested, and adjust the selection probability of the mutation rule.

[0141] In one embodiment, if Fig.12 As shown, the indicator calculation subunit 3041 includes an indicator determination module 30411 and a guidance module 30412.

[0142] The indicator determination module 30411 is used to set the effect indicator corresponding to the test result to 0 when the detection framework to be tested crashes or there is a non-numerical defect in the detection framework to be tested, and the test result indicates that the test input model fails to trigger the corresponding defect; the test result indicates that the test input model successfully triggers the corresponding defect, and the effect indicator corresponding to the test result is set to the average value of the test input tensor value; when the test result indicates an inconsistent detection defect, the effect indicator corresponding to the test result is set to the maximum value of the output tensor difference of the test results under different browsers; the guidance module 30412 is used to guide the selection of mutation rules corresponding to the test input tensor and the test input model according to the effect indicator.

[0143] In one embodiment, if Fig.13 As shown, the adjustment subunit 3042 includes a contribution calculation module 30421 and a control module 30422 .

[0144] Contribution calculation module 30421 is used to calculate the contribution of each mutation rule to the test result of the deep learning framework to be tested by using c1=c0+Δfitness, where c1 and c0 are the overall contribution of the mutation rule to the test result before and after the mutation, and Δfitness=fitness new -fitness old Among them, fitness new Represents the effect index of the newly generated test input model after mutation; fitness old represents the effect index of the test input model before mutation; control module 30422 is used to adopt Control the probability of the mutation rule being selected again, where n represents the total number of mutation rules and p represents the probability of the mutation rule being selected again.

[0145] It should be noted that technicians in the relevant field can clearly understand that the specific implementation process of the above-mentioned deep learning framework testing device 300 and each unit can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and conciseness of the description, it will not be repeated here.

[0146] The deep learning framework testing device 300 can be implemented in the form of a computer program. Fig.14 Runs on the computer device shown.

[0147] See also Fig.14 , Fig.14 500 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a terminal, wherein the terminal may be an electronic device with communication function such as a smart phone, a tablet computer, a laptop computer, a desktop computer, a personal digital assistant, and a wearable device.

[0148] See also Fig.14 The computer device 500 includes a processor 502 , a memory and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0149] The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, which, when executed, may enable the processor 502 to execute a deep learning framework testing method.

[0150] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .

[0151] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a deep learning framework testing method.

[0152] The network interface 505 is used to communicate with other devices over the network. Fig.14 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0153] The processor 502 is used to run the computer program 5032 stored in the memory to implement the following steps:

[0154] A seed tensor is created and mutated to obtain a test input tensor; a seed model is created and mutated to obtain a test input model; a differential test is performed on the deep learning framework to be tested according to the test input tensor and the test input model in different browsers to obtain a test result; and mutation rules corresponding to the seed tensor and the seed model are adjusted according to the test result.

[0155] In one embodiment, when the processor 502 implements the step of creating a seed tensor and mutating it to obtain a test input tensor, the processor 502 specifically implements the following steps:

[0156] A seed tensor is created in the format of (N, C, H, W), where N is the number of batches, C is the number of channels, H is the height, and W is the width; the seed tensor is mutated using a mutation rule of the tensor to obtain a test input tensor, where the mutation rule of the tensor includes at least one of tensor copying, padding, transposing, cropping, and data type conversion.

[0157] In one embodiment, when the processor 502 implements the step of creating a seed model and mutating it to obtain a test input model, the processor 502 specifically implements the following steps:

[0158] A seed model is created, wherein the structure of the seed model is an acyclic chain composed of identity operators; the seed model is mutated using the mutation rule of the model to obtain a test input model, wherein the mutation rule includes at least one of zero-dimensional tensorization, non-shift transposition, dimensionality reduction averaging of height-width dimensions, transposition-based operator rearrangement, maximum pooling-based operator rearrangement, equivalent separable convolution, equivalent convolution-normalization-activation block, and random operator.

[0159] In one embodiment, when the processor 502 implements the step of performing differential testing on the deep learning framework to be tested according to the test input tensor and the test input model under different browsers to obtain the test result, the processor 502 specifically implements the following steps:

[0160] Under different browsers, the test input tensor and the test input model are used as input, and the deep learning framework to be tested is used as a test object, and differential testing is performed to obtain test results.

[0161] In one embodiment, when the processor 502 implements the step of adjusting the mutation rule corresponding to the seed tensor and the seed model according to the test result, the processor 502 specifically implements the following steps:

[0162] Calculate the effect index corresponding to the test result, and guide the selection of mutation rules corresponding to the test input tensor and the test input model; record the contribution of each mutation rule to the test result of the deep learning framework to be tested, and adjust the selection probability of the mutation rule.

[0163] In one embodiment, the processor 502 implements the following steps when calculating the effect index corresponding to the test result and guiding the mutation rule selection step corresponding to the test input tensor and the test input model:

[0164] When the detection framework to be tested crashes or there is a non-numerical defect in the detection framework to be tested, the test result indicates that the test input model fails to trigger the corresponding defect, and the effect indicator corresponding to the test result is set to 0; the test result indicates that the test input model successfully triggers the corresponding defect, and the effect indicator corresponding to the test result is set to the average value of the test input tensor value; when the test result indicates an inconsistent detection defect, the effect indicator corresponding to the test result is set to the maximum value of the output tensor difference of the test results under different browsers; the selection of mutation rules corresponding to the test input tensor and the test input model is guided by the effect indicator.

[0165] In one embodiment, when the processor 502 implements the step of recording the contribution of each mutation rule to the test result of the deep learning framework to be tested and adjusting the selection probability of the mutation rule, the processor 502 specifically implements the following steps:

[0166] The contribution of each mutation rule to the test result of the deep learning framework to be tested is calculated using c1=c0+Δfitness, where c1 and c0 are the overall contributions of the mutation rule to the test result before and after the mutation, and Δfitness=fitness new -fitness old Among them, fitness new Represents the effect index of the newly generated test input model after mutation; fitness old Represents the effect index of the test input model before mutation; according to the contribution Control the probability of the mutation rule being selected again, where n represents the total number of mutation rules and p represents the probability of the mutation rule being selected again.

[0167] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0168] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0169] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein when the computer program is executed by a processor, the processor executes the following steps:

[0170] A seed tensor is created and mutated to obtain a test input tensor; a seed model is created and mutated to obtain a test input model; a differential test is performed on the deep learning framework to be tested according to the test input tensor and the test input model in different browsers to obtain a test result; and mutation rules corresponding to the seed tensor and the seed model are adjusted according to the test result.

[0171] In one embodiment, when the processor executes the computer program to implement the step of creating a seed tensor and mutating it to obtain a test input tensor, the processor specifically implements the following steps:

[0172] A seed tensor is created in the format of (N, C, H, W), where N is the number of batches, C is the number of channels, H is the height, and W is the width; the seed tensor is mutated using a mutation rule of the tensor to obtain a test input tensor, where the mutation rule of the tensor includes at least one of tensor copying, padding, transposing, cropping, and data type conversion.

[0173] In one embodiment, when the processor executes the computer program to implement the step of creating a seed model and mutating it to obtain a test input model, the processor specifically implements the following steps:

[0174] A seed model is created, wherein the structure of the seed model is an acyclic chain composed of identity operators; the seed model is mutated using the mutation rule of the model to obtain a test input model, wherein the mutation rule includes at least one of zero-dimensional tensorization, non-shift transposition, dimensionality reduction averaging of height-width dimensions, transposition-based operator rearrangement, maximum pooling-based operator rearrangement, equivalent separable convolution, equivalent convolution-normalization-activation block, and random operator.

[0175] In one embodiment, when the processor executes the computer program to implement the step of performing differential testing on the deep learning framework to be tested according to the test input tensor and the test input model under different browsers to obtain a test result, the processor specifically implements the following steps:

[0176] Under different browsers, the test input tensor and the test input model are used as input, and the deep learning framework to be tested is used as a test object, and differential testing is performed to obtain test results.

[0177] In one embodiment, when the processor executes the computer program to implement the step of adjusting the mutation rule corresponding to the seed tensor and the seed model according to the test result, the processor specifically implements the following steps:

[0178] Calculate the effect index corresponding to the test result, and guide the selection of mutation rules corresponding to the test input tensor and the test input model; record the contribution of each mutation rule to the test result of the deep learning framework to be tested, and adjust the selection probability of the mutation rule.

[0179] In one embodiment, when the processor executes the computer program to implement the step of calculating the effect index corresponding to the test result and guiding the mutation rule selection step corresponding to the test input tensor and the test input model, the processor specifically implements the following steps:

[0180] When the detection framework to be tested crashes or there is a non-numerical defect in the detection framework to be tested, the test result indicates that the test input model fails to trigger the corresponding defect, and the effect indicator corresponding to the test result is set to 0; the test result indicates that the test input model successfully triggers the corresponding defect, and the effect indicator corresponding to the test result is set to the average value of the test input tensor value; when the test result indicates an inconsistent detection defect, the effect indicator corresponding to the test result is set to the maximum value of the output tensor difference of the test results under different browsers; the selection of mutation rules corresponding to the test input tensor and the test input model is guided by the effect indicator.

[0181] In one embodiment, when the processor executes the computer program to implement the step of recording the contribution of each mutation rule to the test result of the deep learning framework to be tested and adjusting the selection probability of the mutation rule, the processor specifically implements the following steps:

[0182] The contribution of each mutation rule to the test result of the deep learning framework to be tested is calculated using c1=c0+Δfitness, where c1 and c0 are the overall contributions of the mutation rule to the test result before and after the mutation, and Δfitness=fitness new -fitness old Among them, fitness new Represents the effect index of the newly generated test input model after mutation; fitness old Represents the effect index of the test input model before mutation; according to the contribution Control the probability of the mutation rule being selected again, where n represents the total number of mutation rules and p represents the probability of the mutation rule being selected again.

[0183] The storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which are computer-readable storage media that can store program codes.

[0184] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0185] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0186] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0187] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0188] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A deep learning framework testing method, characterized in that: include: Create a seed tensor and mutate it to get the test input tensor; Create a seed model and mutate it to get the test input model; Under different browsers, differential testing is performed on the deep learning framework to be tested according to the test input tensor and the test input model to obtain a test result; The mutation rules corresponding to the seed tensor and the seed model are adjusted according to the test results.

2. The deep learning framework testing method according to claim 1, characterized in that: The step of creating a seed tensor and mutating it to obtain a test input tensor includes: Create a seed tensor in the format of (N, C, H, W), where N is the number of batches, C is the number of channels, H is the height, and W is the width; The seed tensor is mutated using a mutation rule of the tensor to obtain a test input tensor, wherein the mutation rule of the tensor includes at least one of tensor copying, padding, transposing, clipping, and data type conversion.

3. The deep learning framework testing method according to claim 1, characterized in that: The step of creating a seed model and mutating it to obtain a test input model includes: Creating a seed model, wherein the structure of the seed model is an acyclic chain consisting of identity operators; The seed model is mutated using a mutation rule of the model to obtain a test input model, wherein the mutation rule includes at least one of zero-dimensional tensorization, non-shift transposition, dimensionality reduction averaging of height-width dimensions, transposition-based operator rearrangement, maximum pooling-based operator rearrangement, equivalent separable convolution, equivalent convolution-normalization-activation block, and random operator.

4. The deep learning framework testing method according to claim 1, characterized in that: The step of performing differential testing on the deep learning framework to be tested according to the test input tensor and the test input model in different browsers to obtain test results includes: Under different browsers, the test input tensor and the test input model are used as input, and the deep learning framework to be tested is used as a test object, and differential testing is performed to obtain test results.

5. The deep learning framework testing method according to claim 1, characterized in that: The step of adjusting the mutation rules corresponding to the seed tensor and the seed model according to the test result includes: Calculate the effect index corresponding to the test result, and guide the selection of mutation rules corresponding to the test input tensor and the test input model; Record the contribution of each mutation rule to the test results of the deep learning framework to be tested, and adjust the selection probability of the mutation rule.

6. The deep learning framework testing method according to claim 5, characterized in that: The calculating the effect index corresponding to the test result and guiding the selection of mutation rules corresponding to the test input tensor and the test input model include: When the detection framework to be tested crashes or there is a non-numerical defect in the detection framework to be tested, the test result indicates that the test input model fails to trigger the corresponding defect, and the effect indicator corresponding to the test result is set to 0; the test result indicates that the test input model successfully triggers the corresponding defect, and the effect indicator corresponding to the test result is set to the average value of the test input tensor value; when the test result indicates a detection inconsistency defect, the effect indicator corresponding to the test result is set to the maximum value of the output tensor difference of the test results under different browsers; The selection of mutation rules corresponding to the test input tensor and the test input model is guided by the effect index.

7. The deep learning framework testing method according to claim 5, characterized in that: The recording of the contribution of each mutation rule to the test result of the deep learning framework to be tested and adjusting the selection probability of the mutation rule include: The contribution of each mutation rule to the test result of the deep learning framework to be tested is calculated using c1=c0+Δfitness, where c1 and c0 are the overall contributions of the mutation rule to the test result before and after the mutation, and Δfitness=fitness new -fitness old Among them, fitness new Represents the effect index of the newly generated test input model after mutation; fitness old Represents the effect index of the test input model before mutation; According to the contribution adopted Control the probability of the mutation rule being selected again, where n represents the total number of mutation rules and p represents the probability of the mutation rule being selected again.

8. A deep learning framework testing device, characterized in that: include: A test input tensor generation unit, which is used to create a seed tensor and mutate it to obtain a test input tensor; A test input model generation unit, used to create a seed model and perform mutation to obtain a test input model; A defect detection unit, used for performing differential testing on the deep learning framework to be tested according to the test input tensor and the test input model under different browsers to obtain a test result; The heuristic guidance unit is used to adjust the mutation rules corresponding to the seed tensor and the seed model according to the test results.

9. A computer device, characterized in that: The computer device comprises a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images