Methods, devices, and electronic equipment for detecting code defects in dynamic languages

Abstract

This application provides a method, apparatus, and electronic device for detecting code defects in dynamic languages. The method includes: acquiring the source code of the dynamic language; performing syntax tree analysis on the source code to construct an original syntax tree; identifying variables marked as functions in the original syntax tree and tracing the functions called by the variables; updating the original syntax tree according to the functions called by the variables to obtain an updated syntax tree; and analyzing whether the source code has defects based on the updated syntax tree. This approach can improve the accuracy of code defect detection by constructing a complete data flow through tracking the dynamic calls of functions.

Description

Technical Field

[0001] This application relates to the field of source code defect detection technology, and in particular to a method and apparatus for detecting code defects in dynamic languages, electronic devices, and computer-readable storage media. Background Technology

[0002] With the advancement of software design technologies, the expansion of software or system scale, and the increasing complexity of software products, software security issues are becoming increasingly apparent. To ensure software quality, software testing is of paramount importance.

[0003] Source code defect detection technology refers to analyzing and modeling the source code of a program, simulating the execution flow of the source code, examining API calls, and identifying problematic code. The common approach is to construct an abstract syntax tree (AST) from the source code and simulate the relationships between its members. Based on this, the code is simulated for execution to find potential defects.

[0004] Traditional defect detection methods can handle the construction of polluted data streams well for strongly typed languages, but they cannot construct accurate data streams for dynamic languages ​​such as Python that can use functions as variables, resulting in data stream interruptions and missed defects. Summary of the Invention

[0005] This application provides a method for detecting code defects in dynamic languages, which constructs a complete data stream to facilitate defect detection.

[0006] This application provides a method for detecting code defects in a dynamic language, the method comprising:

[0007] Obtain the source code of a dynamic language;

[0008] Perform syntax tree analysis on the source code to construct the original syntax tree;

[0009] Identify variables marked as functions in the original syntax tree and trace the functions called by those variables;

[0010] The original syntax tree is updated based on the function called by the variable to obtain the updated syntax tree;

[0011] Based on the updated syntax tree, analyze whether the source code has defects.

[0012] In one embodiment, identifying variables marked as functions in the original syntax tree and tracing the functions called by the variables includes:

[0013] Based on the labeling information of each variable, determine the variables marked as functions in the original syntax tree;

[0014] Iterate through all assignment statements and trace the functions called by the variables.

[0015] In one embodiment, updating the original syntax tree based on the function called by the variable to obtain an updated syntax tree includes:

[0016] Based on the function called by the variable, the variable in the original syntax tree is updated to the function, thus obtaining the updated syntax tree.

[0017] In one embodiment, the method further includes:

[0018] If the function called by the variable is an unknown function, predict the type information of the calling object of the variable;

[0019] Based on the type information of the calling object of the variable, the unknown function is predicted to be the target function corresponding to the type information.

[0020] In one embodiment, updating the original syntax tree based on the function called by the variable to obtain an updated syntax tree includes:

[0021] Based on the predicted objective function, the variables in the original syntax tree are updated to the objective function to obtain the updated syntax tree.

[0022] In one embodiment, predicting the type information of the calling object of the variable if the function called by the variable is an unknown function includes:

[0023] Based on the context of the variable, identify all function calls to the variable and collect all function names;

[0024] Based on the type information corresponding to the function name stored in the hash table, the type information of the calling object of the variable is determined.

[0025] In one embodiment, before determining the type information of the calling object of the variable based on the type information corresponding to the function name stored in the hash table, the method further includes:

[0026] Collect all type information and all function names, using the function name as the key and the type information to which the function name belongs as the value, and store the function name and corresponding type information as a hash table.

[0027] This application also provides a code defect detection device for dynamic languages, including:

[0028] The code acquisition module is used to acquire the source code of dynamic languages;

[0029] The syntax analysis module is used to perform syntax tree analysis on the source code and construct the original syntax tree;

[0030] The function tracing module is used to identify variables marked as functions in the original syntax tree and to trace the functions called by those variables;

[0031] The syntax tree update module is used to update the original syntax tree according to the function called by the variable, so as to obtain an updated syntax tree;

[0032] The defect detection module is used to analyze whether the source code has defects based on the updated syntax tree.

[0033] This application also provides an electronic device, the electronic device comprising:

[0034] processor;

[0035] Memory used to store processor-executable instructions;

[0036] The processor is configured to execute the code defect detection method for the aforementioned dynamic language.

[0037] This application also provides a computer-readable storage medium storing a computer program that can be executed by a processor to perform the above-described method for detecting code defects in dynamic languages.

[0038] The technical solution provided in the above embodiments of this application constructs an original syntax tree by performing syntax tree analysis on the source code; identifies variables marked as functions in the original syntax tree and tracks the functions called by the variables; updates the original syntax tree according to the functions called by the variables to obtain an updated syntax tree; and analyzes whether there are defects in the source code based on the updated syntax tree. Thus, by tracking the dynamic calls of functions, a complete data flow can be constructed, improving the accuracy of code defect detection. Attached Figure Description

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments of the present application.

[0040] Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0041] Figure 2 This is a flowchart illustrating a method for detecting code defects in a dynamic language according to an embodiment of this application;

[0042] Figure 3 yes Figure 2 A flowchart illustrating a code defect detection method for a dynamic language based on a corresponding embodiment;

[0043] Figure 4 yes Figure 3 Detailed flowchart of step S310 in the corresponding embodiment;

[0044] Figure 5 This is a block diagram of a code defect detection device for a dynamic language provided in an embodiment of this application. Detailed Implementation

[0045] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0046] Similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0047] Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 100 can be used to execute the code defect detection method for dynamic languages ​​provided in an embodiment of this application. Figure 1 As shown, the electronic device 100 includes: one or more processors 102 and one or more memories 104 storing processor-executable instructions. The processors 102 are configured to execute the code defect detection method for dynamic languages ​​provided in the following embodiments of this application.

[0048] The processor 102 may be a gateway, a smart terminal, or a device that includes a central processing unit (CPU), a graphics processing unit (GPU), or other forms of processing units with data processing capabilities and / or instruction execution capabilities. It can process data from other components in the electronic device 100 and control other components in the electronic device 100 to perform desired functions.

[0049] The memory 104 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 102 may execute the program instructions to implement the code defect detection method for dynamic languages ​​described below. Various application programs and various data may also be stored in the computer-readable storage medium, such as various data used and / or generated by the application programs.

[0050] In one embodiment, Figure 1 The electronic device 100 may also include an input device 106, an output device 108, and a data acquisition device 110, which are interconnected via a bus system 112 and / or other forms of connection mechanisms (not shown). It should be noted that... Figure 1 The components and structure of the electronic device 100 shown are merely exemplary and not limiting; the electronic device 100 may also have other components and structures as needed.

[0051] The input device 106 may be a device used by a user to input commands, and may include one or more of a keyboard, mouse, microphone, and touchscreen. The output device 108 may output various information (e.g., images or sounds) to the outside (e.g., a user), and may include one or more of a display, speaker, etc. The data acquisition device 110 may acquire images of objects and store the acquired images in the memory 104 for use by other components. For example, the data acquisition device 110 may be a camera.

[0052] In one embodiment, the components in the example electronic device 100 used to implement the code defect detection method for dynamic languages ​​in the embodiments of this application can be integrated or distributed. For example, the processor 102, memory 104, input device 106 and output device 108 can be integrated into one unit, while the data acquisition device 110 can be separated.

[0053] In one embodiment, the example electronic device 100 for implementing the code defect detection method for dynamic languages ​​in the embodiments of this application can be implemented as a smart terminal such as a smartphone, tablet computer, smartwatch, or in-vehicle device.

[0054] Figure 2This is a flowchart illustrating a code defect detection method for dynamic languages ​​provided in an embodiment of this application. The method can be executed by the aforementioned electronic device, such as... Figure 2 As shown, the method includes steps S210-S250.

[0055] Step S210: Obtain the source code of the dynamic language.

[0056] Source code refers to an uncompiled text file written according to certain programming language specifications; it is a series of human-readable computer language instructions.

[0057] Dynamic languages ​​are languages ​​where the program's structure can be changed at runtime: new functions can be introduced, existing functions can be deleted, and so on. Dynamic languages ​​determine data types at runtime. Variables do not require type declarations before use; typically, the variable's type is the type of the value assigned to it. Dynamic languages ​​include Python, SQL, JavaScript, PHP, and others. Functions in dynamic languages ​​like Python can be used as variables. When performing source code defect detection, traditional detection methods cannot construct data flow graphs for dynamic calls. The embodiments described below in this application employ higher-order function analysis, which can trace the source of functions, thereby accurately tracking dynamic calls.

[0058] Step S220: Perform syntax tree analysis on the source code to construct the original syntax tree.

[0059] The raw syntax tree (AST) is an abstract syntax tree generated by performing syntax tree analysis on the source code. To distinguish it from the updated syntax tree described below, it will be referred to as the raw syntax tree here. An Abstract Syntax Tree (AST), or simply a syntax tree, is an abstract representation of the syntactic structure of source code. It represents the syntactic structure of a programming language in a tree-like structure, where each node represents a structure in the source code. The syntax is described as "abstract" because it doesn't represent every detail of the actual syntax. For example, nested parentheses are implicit in the tree structure and not presented as nodes; while conditional jump statements like `if-condition-then` can be represented using nodes with two branches.

[0060] In one embodiment, the corresponding abstract syntax tree can be generated by performing lexical and syntactic analysis on the source code and utilizing the ast module in the Python standard library. The Python standard library is distributed with the Python language and contains many built-in modules that provide system-level access to functions; the ast module is a built-in module in the Python standard library that helps parse Python's abstract syntax.

[0061] Step S230: Identify variables marked as functions in the original syntax tree and trace the functions called by the variables.

[0062] In the syntax tree, variables whose values ​​may be functions are specially marked. Therefore, based on the marking information of each variable, the variables marked as functions in the original syntax tree can be identified. All assignment statements are traversed to trace the functions called by those variables. For example, the functions might be negations, absolute values, integer functions, etc.

[0063] In dynamic languages, a function itself can be a value. Functions can be assigned to variables, passed as arguments to other functions, and returned as return values ​​by other functions. This allows us to trace the function passing process and ultimately determine which function was called by the variable.

[0064] Step S240: Update the original syntax tree according to the function called by the variable to obtain the updated syntax tree.

[0065] After tracing the called functions, the structure of the original syntax tree can be modified to construct a complete data flow. Specifically, based on the functions called by the variables, the variables in the original syntax tree can be updated to the functions called by those functions, resulting in the updated syntax tree. For distinction, the abstract syntax tree resulting from updating the original syntax tree can be called the updated syntax tree.

[0066] Step S250: Based on the updated syntax tree, analyze whether there are defects in the source code.

[0067] Here, a defect refers to an error that causes security, stability, or recoverability issues in the software, or an error that produces unexpected results. In one embodiment, defect detection can be implemented according to preset defect detection rules.

[0068] In another embodiment, all nodes of the generated updated syntax tree can be converted into tokens according to the different levels of coarseness of the information contained in the nodes; using word embedding technology, the tokens of all nodes are mapped to the word vector space so that they can be input into the neural network; the updated syntax tree represented in word vector form is traversed to obtain the updated syntax tree node sequence represented in word vector form; a long short-term memory neural network is constructed, and the sequence of abstract syntax tree nodes represented by word vectors is sequentially input into the long short-term memory neural network to extract the semantic features of the abstract syntax tree; the extracted semantic features are input into a logistic regression classifier to analyze the probability that the source code to be detected has defects.

[0069] The technical solution provided in the above embodiments of this application constructs an original syntax tree by performing syntax tree analysis on the source code; identifies variables marked as functions in the original syntax tree and tracks the functions called by the variables; updates the original syntax tree according to the functions called by the variables to obtain an updated syntax tree; and analyzes whether there are defects in the source code based on the updated syntax tree. Thus, by tracking the dynamic calls of functions, a complete data flow can be constructed, improving the accuracy of code defect detection.

[0070] In one embodiment, such as Figure 3 As shown, the method provided in this application embodiment further includes steps S310-S320.

[0071] Step S310: If the function called by the variable is an unknown function, predict the type information of the calling object of the variable.

[0072] In dynamic languages, unless actually executed, it's possible to not know what a function call actually invokes; such functions are called unknown functions, or unrecognized functions. In object-oriented languages, functions are bound to an object; calling the function is equivalent to invoking a method of that object. Therefore, unknown functions can be predicted by guessing the type of the object being called.

[0073] The type information can include integer type, floating-point type, string type, boolean type, list type, and dictionary type.

[0074] In one embodiment, such as Figure 4 As shown, step S310 specifically includes steps S311-S312.

[0075] Step S311: Based on the context of the variable, find all function calls to the variable and collect all function names.

[0076] Step S312: Determine the type information of the calling object of the variable based on the type information corresponding to the function name stored in the hash table.

[0077] In one embodiment, all type information and all function names can be collected, and the function names and corresponding type information can be stored as a hash table, with the function names as keys and the type information to which the function names belong as values.

[0078] Afterwards, based on the function names collected in step S311, the hash table can be traversed to find the type information that completely matches these function names, which is the type information of the called object of the guessed variable.

[0079] Step S320: Based on the type information of the calling object of the variable, predict that the unknown function is the target function corresponding to the type information.

[0080] Here, the objective function refers to the possible function types of the unknown function. Based on the type information of the calling object of the variable, the unknown function can be updated to its possible function types, thus enabling the guessing of the call to the unknown function.

[0081] Based on the above embodiments, the variables in the original syntax tree can be updated to the target function according to the predicted target function, resulting in the updated syntax tree. Thus, by guessing the unknown function, the target function can be obtained, facilitating the construction of a complete data stream for defect detection.

[0082] The following are embodiments of the apparatus described in this application, which can be used to execute the code defect detection method for dynamic languages ​​described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the code defect detection method for dynamic languages ​​described in this application.

[0083] Figure 5 This is a block diagram of a code defect detection device for a dynamic language provided in an embodiment of this application. Figure 5 As shown, the device includes: a code acquisition module 510, a syntax analysis module 520, a function tracing module 530, a syntax tree update module 540, and a defect detection module 550.

[0084] Code acquisition module 510 is used to acquire the source code of dynamic languages;

[0085] The syntax analysis module 520 is used to perform syntax tree analysis on the source code and construct the original syntax tree;

[0086] The function tracing module 530 is used to identify variables marked as functions in the original syntax tree and to trace the functions called by the variables;

[0087] The syntax tree update module 540 is used to update the original syntax tree according to the function called by the variable, so as to obtain an updated syntax tree;

[0088] The defect detection module 550 is used to analyze whether there are defects in the source code based on the updated syntax tree.

[0089] In one embodiment, the function tracing module 530 is specifically used to determine the variables marked as functions in the original syntax tree based on the marking information of each variable; and to traverse all assignment statements to trace the functions called by the variables.

[0090] In one embodiment, the syntax tree update module 540 is specifically used to: update the variable in the original syntax tree to the function called by the variable, thereby obtaining the updated syntax tree.

[0091] In one embodiment, the apparatus further includes: a type prediction module, configured to predict the type information of the calling object of the variable if the function called by the variable is an unknown function; and a function prediction module, configured to predict that the unknown function is the target function corresponding to the type information based on the type information of the calling object of the variable.

[0092] In one embodiment, the syntax tree update module 540 is specifically used to: obtain the target function based on the prediction, update the variables in the original syntax tree to the target function, and obtain the updated syntax tree.

[0093] In one embodiment, the type prediction module is specifically used to: find all function calls to the variable based on the context of the variable, and collect all function names; and determine the type information of the calling object of the variable based on the type information corresponding to the function names stored in the hash table.

[0094] In one embodiment, the type prediction module is further configured to: before determining the type information of the calling object of the variable based on the type information corresponding to the function name stored in the hash table, collect all type information and all function names, and store the function name and corresponding type information as a hash table with the function name as the key and the type information to which the function name belongs as the value.

[0095] The implementation process of the functions and roles of each module in the above-mentioned device is detailed in the implementation process of the corresponding steps in the above-mentioned dynamic language code defect detection method, and will not be repeated here.

[0096] The apparatuses and methods disclosed in the several embodiments provided in this application can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatuses, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0097] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0098] If a function is implemented as a software module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

Claims

1. A method for detecting code defects in a dynamic language, characterized in that, The method includes: Obtain the source code of a dynamic language; Perform syntax tree analysis on the source code to construct the original syntax tree; Identify variables marked as functions in the original syntax tree and trace the functions called by those variables; The original syntax tree is updated based on the function called by the variable to obtain the updated syntax tree; Based on the updated syntax tree, analyze whether the source code has any defects; The method further includes: if the function called by the variable is an unknown function, predicting the type information of the calling object of the variable; Based on the type information of the calling object of the variable, the unknown function is predicted to be the target function corresponding to the type information.

2. The method according to claim 1, characterized in that, The step of identifying variables marked as functions in the original syntax tree and tracing the functions called by those variables includes: Based on the labeling information of each variable, determine the variables marked as functions in the original syntax tree; Iterate through all assignment statements and trace the functions called by the variables.

3. The method according to claim 1, characterized in that, The step of updating the original syntax tree based on the function called by the variable to obtain the updated syntax tree includes: Based on the function called by the variable, the variable in the original syntax tree is updated to the function, thus obtaining the updated syntax tree.

4. The method according to claim 1, characterized in that, The step of updating the original syntax tree based on the function called by the variable to obtain the updated syntax tree includes: Based on the predicted objective function, the variables in the original syntax tree are updated to the objective function to obtain the updated syntax tree.

5. The method according to claim 1, characterized in that, If the function called by the variable is an unknown function, predicting the type information of the calling object of the variable includes: Based on the context of the variable, identify all function calls to the variable and collect all function names; Based on the type information corresponding to the function name stored in the hash table, the type information of the calling object of the variable is determined.

6. The method according to claim 5, characterized in that, Before determining the type information of the calling object of the variable based on the type information corresponding to the function name stored in the hash table, the method further includes: Collect all type information and all function names, using the function name as the key and the type information to which the function name belongs as the value, and store the function name and corresponding type information as a hash table.

7. A code defect detection device for a dynamic language, characterized in that, include: The code acquisition module is used to acquire the source code of dynamic languages; The syntax analysis module is used to perform syntax tree analysis on the source code and construct the original syntax tree; The function tracing module is used to identify variables marked as functions in the original syntax tree and trace the functions called by the variables; wherein, if the function called by the variable is an unknown function, the module predicts the type information of the calling object of the variable; based on the type information of the calling object of the variable, the module predicts that the unknown function is the target function corresponding to the type information. The syntax tree update module is used to update the original syntax tree according to the function called by the variable, so as to obtain an updated syntax tree; The defect detection module is used to analyze whether the source code has defects based on the updated syntax tree.

8. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store processor-executable instructions; The processor is configured to execute the code defect detection method for dynamic languages ​​according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that can be executed by a processor to perform the code defect detection method for dynamic languages ​​as described in any one of claims 1-6.

Images