Cross-framework API (Application Program Interface) defect detection method, system and related device

Abstract

The invention discloses a cross-framework API (Application Program Interface) defect detection method and system and a related device. The method comprises the following steps: acquiring an API defect code, and determining a defect parameter in the API defect code based on the API defect code; traversing and searching similar APIs most similar to the API defect codes from all APIs under each to-be-tested deep learning framework by using a preset similarity algorithm to obtain a similar API set; for each similar API in the similar API set, generating a test case corresponding to the similar API by utilizing a parameter matching relationship between each similar API and the API defect code and defect parameters in the API defect code; and performing fuzzy testing on the similar APIs by utilizing the test cases to obtain defect detection results of the similar APIs. According to the scheme provided by the invention, cross-framework large-scale API defect testing can be realized.

Description

Technical Field

[0001] The present invention relates to the field of defect detection technology, and in particular to a cross-framework API defect detection method, system and related devices. Background Art

[0002] Deep learning technology has been widely used in various fields. Among them, deep learning frameworks play a vital role, providing developers with a series of simple and flexible APIs to build, train and deploy various complex deep learning models and applications. However, with the popularization of deep learning technology, security issues have gradually emerged. As the core of the entire deep learning system, the security of the deep learning framework is particularly important. Once there are defects in the API of the deep learning framework, it will affect the deep learning models trained using these frameworks, thereby affecting various fields where these models are applied, causing serious harm. Therefore, it is crucial to ensure the security of the deep learning framework API for building deep learning models.

[0003] Deep learning framework defects mainly refer to defects or loopholes in the interfaces or functions of the deep learning framework, which lead to security risks or functional abnormalities. At present, there are two main methods for detecting deep learning framework defects: model-based fuzz testing and API-based fuzz testing. Model-based fuzz testing uses deep neural network models as test cases for fuzz testing, and performs differential testing on the model output results to detect defects in the deep learning framework; API-based fuzz testing collects or generates test cases for the API of the deep learning framework, and then performs differential testing on the running results of the API to detect defects in the deep learning framework.

[0004] However, the above two methods have the following defects: Although model-based fuzz testing helps to find defects in deep learning model training and verification, it has the problem of insufficient code coverage and requires a large number of different deep learning models. API-based fuzz testing requires accurate constraint information to generate higher quality test cases. However, due to the large number of deep learning framework APIs and complex relationships, it is difficult to effectively test the deep learning framework APIs. In addition, current methods fail to fully utilize the large number of APIs with similar functions in different deep learning frameworks, and these APIs share the same error features for defect detection. Summary of the invention

[0005] In order to solve the above technical problems, an embodiment of the present invention provides a cross-framework API defect detection method, system and related devices.

[0006] The technical solution of the embodiment of the present invention is achieved as follows:

[0007] An embodiment of the present invention provides a cross-framework API defect detection method, the method comprising: obtaining an API defect code, and determining defect parameters in the API defect code based on the API defect code; using a preset similarity algorithm to traverse and search for similar APIs that are most similar to the API defect code from all APIs under each deep learning framework to be tested, and obtain a similar API set; for each similar API in the similar API set, using a parameter matching relationship between each similar API and the API defect code, and the defect parameters in the API defect code to generate a test case corresponding to the similar API; using the test case to perform fuzzy testing on the similar API to obtain a defect detection result of the similar API.

[0008] In one embodiment, defect parameters in the API defect code are determined based on the API defect code, including: dynamically running the API defect code to obtain a parameter list and corresponding parameter values ​​in the API defect code; dividing the parameter list into a required parameter list and an optional parameter list through API document constraint analysis; the required parameter list is the parameters that must be passed when the API is called; the optional parameter list is the other parameters in the parameter list except the required parameters; for each optional parameter in the optional parameter list, iteratively testing whether each optional parameter is a parameter that causes a defect; deleting the optional parameters that do not cause a defect from the parameter list to obtain a defect parameter list.

[0009] In one embodiment, a preset similarity algorithm is used to traverse and search for similar APIs that are most similar to the API defect code from all APIs under each deep learning framework to be tested, and a similar API set is obtained, including: for each API under each deep learning framework to be tested, the corresponding API document information is obtained; the API document information includes an API function signature, an API sample code, and an API function description; the API document information is represented by word vector embedding to obtain the word vector embedding corresponding to the API document information; the preset similarity algorithm is used to calculate the similarity between the word vector embedding of the API and the word vector embedding corresponding to the API defect code; according to the similarity of each API, the similar API that is most similar to the API defect code is searched from all APIs under each deep learning framework to be tested; similar APIs under the deep learning framework to be tested are obtained to form a similar API set.

[0010] In one embodiment, the preset similarity algorithm is:

[0011]

[0012] in, is the similarity between word vector embeddings, For API i The word vector embedding of For API j The word vector embedding of For API i The model of the word vector embedding, For API j The modulus of the word vector embedding.

[0013] In one embodiment, for each similar API in the similar API set, a test case corresponding to the similar API is generated by utilizing the parameter matching relationship between each similar API and the API defect code, as well as the defect parameters in the API defect code, including: for each similar API in the similar API set, a preset parameter matching module is used to perform automatic parameter matching on the parameter list corresponding to the API defect code and the parameter list corresponding to each similar API to determine the parameter pairing relationship between the API defect code and each similar API; based on the parameter pairing relationship, a test case corresponding to each similar API is generated by utilizing the defect parameters corresponding to the API defect code.

[0014] In one embodiment, after obtaining a similar API set, the method further includes: for other APIs outside the similar API set, obtaining parameter information of the other APIs, the parameter information including parameter type, parameter value and parameter shape; searching for API defects whose parameter information is similar to the parameter information of the other APIs from a preset defect database; and generating test cases for the other APIs using the parameter information of the API defects.

[0015] In one embodiment, the test case is used to perform fuzz testing on the same type of API to obtain a defect detection result of the same type of API, including: during the fuzz testing process, if a non-runtime error message appears or the test results are inconsistent, determining that the defect detection result of the same type of API is a defect.

[0016] An embodiment of the present invention also provides a cross-framework API defect detection system, which includes: an acquisition module, used to obtain an API defect code, and determine the defect parameters in the API defect code based on the API defect code; a similarity algorithm module, used to use a preset similarity algorithm to traverse and search for similar APIs that are most similar to the API defect code from all APIs under each deep learning framework to be tested, and obtain a similar API set; a fuzzy testing module, used to generate a test case corresponding to each similar API in the similar API set by using the parameter matching relationship between each similar API and the API defect code, and the defect parameters in the API defect code; and use the test case to perform fuzzy testing on the similar API to obtain a defect detection result of the similar API.

[0017] An embodiment of the present invention further provides an electronic device, comprising: a processor and a memory for storing a computer program that can be run on the processor; wherein the processor executes the steps of any one of the above methods when running the computer program.

[0018] An embodiment of the present invention further provides a storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0019] The cross-framework API defect detection method, system and related device provided by the embodiment of the present invention obtain API defect code, determine the defect parameters in the API defect code based on the API defect code; use a preset similarity algorithm to traverse and find the same API that is most similar to the API defect code from all APIs under each deep learning framework to be tested, and obtain a set of similar APIs; for each similar API in the set of similar APIs, use the parameter matching relationship between each similar API and the API defect code, and the defect parameters in the API defect code to generate a test case corresponding to the same API; use the test case to perform fuzzy testing on the same API to obtain the defect detection result of the same API. The solution provided by the present invention matches the same APIs in different frameworks according to document similarity, does not require a model conversion rule table, and supports the identification of the same APIs in frameworks that cannot perform model conversion with each other, and can perform defect detection across more deep learning frameworks. And the solution provided by the present invention locates defect parameters, and generates test cases using the same errors in APIs and similar APIs across frameworks, which can more accurately locate potential defects when generating test cases, with higher granularity, and improve the efficiency and coverage of fuzzy testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flowchart of a cross-framework API defect detection method according to an embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram of the structure of a cross-framework API defect detection system according to an embodiment of the present invention;

[0022] Figure 3 1 is a diagram showing the internal structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] Before introducing the present embodiment, the following terms are explained.

[0024] Deep learning framework API: Deep learning framework functional interface, an interface for developing deep learning models, such as the torch.nn.conv2d interface in the pytorch framework, which is a function in pytorch used to perform two-dimensional convolution operations. It can apply a two-dimensional convolution kernel to the input data to generate an output feature vector. This function is often used to construct convolutional layers in convolutional neural networks (CNNs).

[0025] Similar API / equivalent API / peer API / same API: API interfaces with specific similar functions in different deep learning frameworks. For example, the torch.nn.conv2d interface in pytorch and the tf.nn.conv2d interface in tensorflow are a pair of similar APIs, both of which are used to perform two-dimensional convolution operations.

[0026] Deep learning framework defect detection: Through code auditing, manual analysis, fuzz testing, differential testing and other methods, detect whether there are inconsistencies or security issues in the interfaces and algorithm implementations in the deep learning framework. Inconsistency refers to the large difference in the output results of the same deep learning model M on framework A and framework B (in theory, the results of the two should be similar), or the different output results of the same functional interface on framework A and framework B (for example, the two-dimensional convolution function with the same parameters, in theory, the calculation results of each framework should be consistent). In addition, there are other inconsistencies, such as the different calculation results of the same API on the CPU and GPU. Security issues refer to security issues such as memory overflow, crash, and hang in the deep learning framework when calling the API interface.

[0027] Common error characteristics between similar APIs: The vulnerability triggering code of an API in a deep learning framework is modified or directly passed to an API in another framework, causing similar errors. For example, there is a parameter in the AdaptiveMaxPool1d interface in pytorch that does not consider special value situations, causing the code to directly trigger a crash when it is running, torch.nn.AdaptiveMaxPool1d(9132760301568586890)(torch.randn([2,2,2])). In the oneflow deep learning framework, there is an interface oneflow.nn.AdaptiveMaxPool1d that is equivalent to torch.nn.AdaptiveMaxPool1d. And the vulnerability triggering code of pytorch can be directly modified to the syntax format of oneflow, which can also trigger the same error, causing the code to trigger a crash when it is running. The code is as follows oneflow.nn.AdaptiveMaxPool1d(9132760301568586890)(oneflow.randn([2,2,2])). That is, the trigger form and trigger reason are exactly the same, and the trigger code can be directly reproduced on the AdaptiveMaxPool2d and AdaptiveMaxPool3d functional interfaces in the oneflow framework.

[0028] Differential testing: A software testing technique that aims to discover differences in the behavior of software between different implementations. It is often used to test different versions or different implementations of software to ensure that they produce the same output under the same input conditions, or at least behave consistently to some extent. The basic principle of differential testing is to compare the outputs of multiple implementations by executing the same test cases on them. If different outputs are produced under the same input, this may indicate that there is an error in one of the implementations. Differential testing is often used to test the compatibility and consistency between different implementations or versions of compilers, interpreters, libraries, operating systems, and other software systems. In the deep learning framework, differential testing is used to compare the implementations of the same model in different frameworks. Excluding objective reasons such as floating point loss in the deep learning framework, the output results between the two models should be consistent. In addition, it is also used in the deep learning framework that the same API of different frameworks should output the same results under the same input.

[0029] Model conversion and model conversion rule table: In order to improve the efficiency of developing deep learning models, different deep learning frameworks support model conversion functions, which is the process of converting deep learning models from one framework format to another. On the one hand, it can be widely used for cross-framework deployment. In addition, developers with TensorFlow experience can easily reuse PyTorch models using automatic converters. Considering the benefits of cross-framework conversion, many deep learning frameworks have implemented their own converters to support smooth conversion to other popular frameworks. The model conversion rule table is a rule mapping of the converter, which is used to represent equivalent APIs across frameworks and to ensure universality during the model conversion process.

[0030] The present invention will be described in further detail below with reference to the accompanying drawings and embodiments.

[0031] The embodiment of the present invention provides a cross-framework API defect detection method, such as Figure 1 As shown, the method includes:

[0032] Step 101: Obtain an API defect code, and determine defect parameters in the API defect code based on the API defect code;

[0033] Step 102: Using a preset similarity algorithm, traverse and search for similar APIs that are most similar to the API defect code from all APIs under each deep learning framework to be tested, and obtain a similar API set;

[0034] Step 103: For each similar API in the similar API set, generate a test case corresponding to the similar API using the parameter matching relationship between each similar API and the API defect code, and the defect parameters in the API defect code; perform fuzzy testing on the similar API using the test case to obtain the defect detection result of the similar API.

[0035] This embodiment can be applied to similar API defect detection across deep learning frameworks. The deep learning frameworks here can specifically be Tensorflow, Pytorch, Paddle, Mindspore, Oneflow, etc. The function of the deep learning framework is to enable developers to easily build, train and deploy deep learning models, and optimize their performance to meet the needs of various application scenarios. These frameworks have a large number of APIs with similar functions, that is, similar APIs. These APIs have similar code logic and design, so similar errors will also occur. This embodiment is aimed at the common error characteristics existing between cross-framework APIs, using the known API defect codes in each framework, through cross-frame comparison and analysis of functionally equivalent APIs, and using the parameter assignments in these defect codes, passing the parameter values ​​to the matched APIs, and generating test cases, aiming to improve the detection speed and breadth. The advantage of the scheme of this embodiment is that it can widely cover various types of APIs in different frameworks, share the key parameter information that triggers defects, and greatly improve practicality and efficiency.

[0036] Specifically, this embodiment includes the following key technical points:

[0037] (1) Defect parameter positioning

[0038] According to the system architecture design, defect parameter location is mainly based on the defect code crawled by the crawler, and then the parameter list and parameter values ​​in the defect code are obtained by dynamically running the defect code, and then recorded in the database. Then, through the API document constraint analysis, the necessary parameter list (that is, the parameters that must be passed when calling this API) and the optional parameter list are obtained, and then iterative testing is performed, that is, reducing the optional parameters in the defect code to obtain the minimum parameter list that triggers the defect. This list is the located defect parameter list.

[0039] That is, in one embodiment, the defect parameters in the API defect code are determined based on the API defect code, including: dynamically running the API defect code to obtain a parameter list and corresponding parameter values ​​in the API defect code; dividing the parameter list into a necessary parameter list and an optional parameter list through API document constraint analysis; the necessary parameter list is the parameters that must be passed when the API is called; the optional parameter list is the other parameters in the parameter list except the necessary parameters; for each optional parameter in the optional parameter list, iteratively testing whether each optional parameter is a parameter that causes a defect; deleting the optional parameters that do not cause a defect from the parameter list to obtain a defect parameter list.

[0040] (2) Pairing similar APIs across frameworks

[0041] According to the system architecture design, the method of pairing similar APIs across frameworks is as follows:

[0042] First, collect and analyze the API documents of different deep learning frameworks. Use crawlers to extract information such as API function signatures, API sample codes, and API function descriptions, and use this information as API documentation information.

[0043] Second, embed the API document information of different frameworks into word vectors, and then calculate the similarity of the word vector embedding of APIs between different frameworks according to the cosine similarity formula. The API with the highest similarity score is used as the paired API to form a pair of similar APIs. The similarity score calculation process is as follows:

[0044]

[0045] The meaning of this formula is as follows: given a target API in framework M i , its API document information word vector embedding is and a candidate API in framework N j , its API document information word vector embedding is formula Indicates the target API i and candidate APIs j The similarity score between them. The similarity score ranges from [-1, 1]. The closer the score is to 1, the more similar the two APIs are.

[0046]

[0047] The meaning of this formula is as follows: given a target API in framework M i , it will select the target API from the candidate frameworks N i Most similar Specifically, for the target API i , we calculate it with all candidate APIs in framework N j Then select the candidate API with the highest similarity score as Finally, the target API i and paired Form a pair of similar APIs.

[0048] That is, in one embodiment, a preset similarity algorithm is used to traverse and search for similar APIs that are most similar to the API defect code from all APIs under each deep learning framework to be tested, and a similar API set is obtained, including: for each API under each deep learning framework to be tested, the corresponding API document information is obtained; the API document information includes an API function signature, an API sample code, and an API function description; the API document information is represented by word vector embedding to obtain the word vector embedding corresponding to the API document information; the preset similarity algorithm is used to calculate the similarity between the word vector embedding of the API and the word vector embedding corresponding to the API defect code; according to the similarity of each API, the similar API that is most similar to the API defect code is searched from all APIs under each deep learning framework to be tested; similar APIs under the deep learning framework to be tested are obtained to form a similar API set.

[0049] (3) Test case generation based on similar APIs sharing the same error characteristics

[0050] According to the system architecture design, the defect code data set obtained by querying the crawler is divided into two cases:

[0051] Case 1: The dataset contains target APIs for similar API pairs j Due to the implementation differences between different frameworks, even APIs with the same functions may have different names, parameter numbers, parameter types, parameter positions, parameter names, etc. Therefore, it is necessary to identify the target APIs in the same API pair. i and paired Perform parameter matching and transmission. In the previous stage, the API has been obtained j We also collected a large number of similar API pairs and then used the parameter matching function to automatically match the parameters of some similar API pairs. i and paired The argument matching list, such as Arg i ={arg1,arg2,arg3,...}, indicating the target API i Arg is obtained through parameter matching function. j ={arg3,arg1,arg2,...}, indicating the target API i The parameters of the pair Then, based on the target API i The defect location parameter list is passed to Parameters to generate paired Test cases.

[0052] Case 2: The dataset does not contain the target API of the same API pair i For some uncommon APIs, there may be no known defect codes. For similar APIs in this situation, mutation operations can be performed based on parameter types and constraints. That is, the parameter type, parameter value, parameter shape and other information are mutated according to similar defect parameter situations in the database. The defect value of the parameter constraint is directly passed as a parameter.

[0053] That is, in one embodiment, for each similar API in the similar API set, the parameter matching relationship between each similar API and the API defect code, and the defect parameters in the API defect code are used to generate a test case corresponding to the similar API, including: for each similar API in the similar API set, a preset parameter matching module is used to perform automatic parameter matching on the parameter list corresponding to the API defect code and the parameter list corresponding to each similar API to determine the parameter pairing relationship between the API defect code and each similar API; based on the parameter pairing relationship, a test case corresponding to each similar API is generated using the defect parameters corresponding to the API defect code.

[0054] In addition, for other APIs outside the same API set, obtain parameter information of the other APIs, the parameter information including parameter type, parameter value and parameter shape; search for API defects whose parameter information is similar to the parameter information of the other APIs from a preset defect database; and use the parameter information of the API defects to generate test cases for the other APIs.

[0055] (4) Fuzz testing

[0056] After obtaining the generated test cases, fuzz testing is performed. The test prediction of fuzz testing, that is, the criteria for judging whether a defect is triggered, is as follows:

[0057] Crash: After running the test case, non-runtime error messages such as Abort, Segment fault, and Floating PointException appear, which is considered a defect.

[0058] Inconsistency: Based on the principle that the execution results of APIs with the same functions under different frameworks should be consistent under the same input conditions, the test prediction is that if the output results of a pair of APIs are different, it is considered an inconsistency defect.

[0059] That is, in one embodiment, the test case is used to perform fuzz testing on the same type of API to obtain a defect detection result of the same type of API, including: during the fuzz testing process, if a non-runtime error message appears or the test results are inconsistent, determining that the defect detection result of the same type of API is a defect.

[0060] This embodiment scheme is based on the feature that the same errors are shared between similar APIs across frameworks. It collects the defect codes of each framework API, then dynamically runs these codes, locates the defect parameters, saves the relevant information in the database, and then matches similar APIs across frameworks. According to the known API defect codes in the database, matching test cases are constructed and fuzz testing is performed to achieve large-scale API defect testing across frameworks.

[0061] Based on the above content, the specific complete overall process of this embodiment can be as follows:

[0062] First, defect parameter positioning

[0063] First, write a web crawler script to crawl the open source deep learning framework GitHub library, capture the historical issues in the GitHub repository, and capture the defect code and defect description information through HTML parsing.

[0064] Then, the captured defective code is dynamically run. In this stage, the API signature, parameter list, parameter value, parameter type and other information in the defective code are dynamically collected by inserting hook functions into the deep learning framework. After obtaining the API code information, the defective parameters are located. After obtaining the defective parameter list, it is stored in the database together with the API information.

[0065] Second, similar API pairing

[0066] First, we write a web crawler script to obtain the API document information of the deep learning framework and get the API information list of each framework. Then, we traverse each API according to the API list of each framework and pair them with similar APIs. We then obtain the most similar API as the paired API based on the similarity algorithm.

[0067] Third, test case synthesis

[0068] After defect parameter location and pairing of similar APIs, a target API to be tested is given and similar API pairs are queried. If a paired API is found, test cases are generated based on similar APIs sharing the same error features to obtain test cases for paired APIs.

[0069] If no matching API is found, the database will be queried for similar defective parameters based on the parameter information of the API to be tested, and the parameter type, value, shape and other information will be mutated.

[0070] Fourth, fuzz testing

[0071] In fuzz testing, the test cases generated by passing the defect parameter list are directly executed to detect whether a crash or other security issues are triggered, and the newly discovered defect code is updated to the database to find more similar defects. If not, the parameter type, value, shape and other information are mutated according to the defect database. Then, based on the principle that the execution results of the same API under different frameworks should be consistent, it is used as a test prediction to discover potential defects.

[0072] The cross-framework API defect detection method provided by the embodiment of the present invention obtains the API defect code, determines the defect parameters in the API defect code based on the API defect code; uses a preset similarity algorithm to traverse and find the same API that is most similar to the API defect code from all APIs under each deep learning framework to be tested, and obtains a set of similar APIs; for each similar API in the set of similar APIs, uses the parameter matching relationship between each similar API and the API defect code, and the defect parameters in the API defect code to generate a test case corresponding to the same API; uses the test case to perform fuzzy testing on the same API to obtain the defect detection result of the same API. The solution provided by the present invention matches the same APIs in different frameworks according to document similarity, does not require a model conversion rule table, and supports the identification of the same APIs in frameworks that cannot perform model conversion with each other, and can perform defect detection across more deep learning frameworks. And the solution provided by the present invention locates defect parameters, and generates test cases using the same errors in APIs and similar APIs across frameworks, which can more accurately locate potential defects when generating test cases, with higher granularity, and improve the efficiency and coverage of fuzzy testing.

[0073] In order to implement the method of the embodiment of the present invention, the embodiment of the present invention also provides a cross-framework API defect detection system, such as Figure 2 As shown, the cross-framework API defect detection system 200 includes: an acquisition module 201, a similarity algorithm module 202 and a fuzzy testing module 203; wherein,

[0074] An acquisition module 201 is used to acquire an API defect code and determine a defect parameter in the API defect code based on the API defect code;

[0075] A similarity algorithm module 202 is used to use a preset similarity algorithm to traverse and search for similar APIs that are most similar to the API defect code from all APIs under each deep learning framework to be tested, and obtain a similar API set;

[0076] The fuzzy testing module 203 is used to generate a test case corresponding to each similar API in the similar API set by utilizing the parameter matching relationship between each similar API and the API defect code, as well as the defect parameters in the API defect code; and perform fuzzy testing on the similar API using the test case to obtain the defect detection result of the similar API.

[0077] In actual application, the acquisition module 201 , the similarity algorithm module 202 and the fuzzy testing module 203 may be implemented by a processor in the cross-framework API defect detection system 200 .

[0078] It should be noted that: when the above-mentioned system provided in the above-mentioned embodiment is executed, only the division of the above-mentioned program modules is used as an example. In actual application, the above-mentioned processing can be assigned to different program modules as needed, that is, the internal structure of the terminal is divided into different program modules to complete all or part of the above-mentioned processing. In addition, the above-mentioned system provided in the above-mentioned embodiment and the above-mentioned method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0079] In order to implement the method of the embodiment of the present invention, the embodiment of the present invention also provides a computer program product, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of the above method.

[0080] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiment of the present invention, the embodiment of the present invention also provides an electronic device (computer device). Specifically, in one embodiment, the computer device can be a terminal, and its internal structure diagram can be as follows: Figure 3As shown. The computer device includes a processor A01, a network interface A02, a display screen A04, an input device A05 and a memory (not shown in the figure) connected through a system bus. Among them, the processor A01 of the computer device is used to provide computing and control capabilities. The memory of the computer device includes an internal memory A03 and a non-volatile storage medium A06. The non-volatile storage medium A06 stores an operating system B01 and a computer program B02. The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 in the non-volatile storage medium A06. The network interface A02 of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor A01, the method of any one of the above embodiments is implemented. The display screen A04 of the computer device can be a liquid crystal display or an electronic ink display, and the input device A05 of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0081] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0082] The device provided by the embodiment of the present invention includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, the method of any one of the above embodiments is implemented.

[0083] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0084] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0085] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0086] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0087] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0088] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0089] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0090] It can be understood that the memory of the embodiment of the present invention can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memories described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memories.

[0091] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0092] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A cross-framework API defect detection method, characterized in that: The method comprises: Obtaining an API defect code, and determining defect parameters in the API defect code based on the API defect code; Using a preset similarity algorithm, traverse all APIs under each deep learning framework to be tested to find the same type of API that is most similar to the API defect code, and obtain a set of similar APIs; For each similar API in the similar API set, a test case corresponding to the similar API is generated by utilizing the parameter matching relationship between each similar API and the API defect code, as well as the defect parameters in the API defect code; and the similar API is fuzzily tested by utilizing the test case to obtain a defect detection result for the similar API.

2. The cross-framework API defect detection method according to claim 1, characterized in that: Determining a defect parameter in the API defect code based on the API defect code includes: Dynamically run the API defect code to obtain a parameter list and corresponding parameter values ​​in the API defect code; By analyzing the constraints of the API document, the parameter list is divided into a required parameter list and an optional parameter list; the required parameter list is the parameters that must be passed when the API is called; the optional parameter list is the other parameters in the parameter list except the required parameters; For each optional parameter in the optional parameter list, iteratively test whether each optional parameter is a parameter that causes a defect; Optional parameters that do not cause defects are deleted from the parameter list to obtain a defective parameter list.

3. The cross-framework API defect detection method according to claim 1, characterized in that: Using the preset similarity algorithm, we search through all APIs under each deep learning framework to be tested for similar APIs that are most similar to the API defect code, and obtain a similar API set, including: For each API under each deep learning framework to be tested, obtain the corresponding API document information; the API document information includes API function signature, API sample code and API function description; perform word vector embedding representation on the API document information to obtain the word vector embedding corresponding to the API document information; use a preset similarity algorithm to calculate the similarity between the word vector embedding of the API and the word vector embedding corresponding to the API defect code; According to the similarity of each API, searching for the same API that is most similar to the API defect code from all APIs under each deep learning framework to be tested; Obtain similar APIs under the deep learning framework to be tested to form a similar API set.

4. The cross-framework API defect detection method according to claim 3 is characterized in that: The preset similarity algorithm is: in, is the similarity between word vector embeddings, For API i The word vector embedding of For API j The word vector embedding of For API i The model of the word vector embedding, For API j The modulus of the word vector embedding.

5. The cross-framework API defect detection method according to claim 1, characterized in that: For each similar API in the similar API set, a test case corresponding to the similar API is generated by using a parameter matching relationship between each similar API and the API defect code and a defect parameter in the API defect code, including: For each similar API in the similar API set, use a preset parameter matching module to automatically match the parameter list corresponding to the API defect code with the parameter list corresponding to each similar API to determine the parameter pairing relationship between the API defect code and each similar API; According to the parameter pairing relationship, the defect parameters corresponding to the API defect codes are used to generate test cases corresponding to each API of the same type.

6. The cross-framework API defect detection method according to claim 5, characterized in that: After obtaining the same type of API set, the method further includes: For other APIs outside the same API set, obtaining parameter information of the other APIs, the parameter information including parameter type, parameter value and parameter shape; Search a preset defect database for API defects whose parameter information is similar to the parameter information of the other APIs; and generate test cases for the other APIs using the parameter information of the API defects.

7. The cross-framework API defect detection method according to claim 1, characterized in that: Performing fuzz testing on the same type of API using the test case to obtain defect detection results of the same type of API, including: During the fuzz testing process, when a non-runtime error message appears or the test results are inconsistent, it is determined that the defect detection result of the same type of API is defective.

8. A cross-framework API defect detection system, characterized in that: The cross-framework API defect detection system includes: An acquisition module, used for acquiring an API defect code, and determining defect parameters in the API defect code based on the API defect code; A similarity algorithm module is used to use a preset similarity algorithm to traverse all APIs under each deep learning framework to be tested to find the same type of API that is most similar to the API defect code, and obtain a set of similar APIs; A fuzzy testing module is used to generate a test case corresponding to each similar API in the similar API set by utilizing the parameter matching relationship between each similar API and the API defect code, as well as the defect parameters in the API defect code; and to perform fuzzy testing on the similar API using the test case to obtain the defect detection result of the similar API.

9. An electronic device, characterized in that: include: A processor and a memory for storing a computer program that can be run on the processor; wherein, when the processor is used to run the computer program, the steps of the method according to any one of claims 1 to 7 are executed.

10. A storage medium storing a computer program, wherein: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images