Method and device for carrying out vulnerability detection on dynamic type object

Abstract

The embodiment of the invention provides a method for carrying out vulnerability detection on a dynamic type object, the dynamic type object is generated through a target deserialization operation, and the method comprises the steps that a target code file is obtained, and the target code file comprises a target function for carrying out the target deserialization operation; the target function serves as a starting point, the corresponding code file is searched in a backtracking mode along the sequential calling relation of the function, and a plurality of calling chains are obtained and comprise the first calling chain. For any first code file in the first call chain, a corresponding first reference class set is determined according to the first program element and the first reference information in the first code file, and the first reference class set comprises classes for defining the first program element. And obtaining a first recognizable class set based on a set of the reference class sets corresponding to the code files on the first call chain, determining a class subset containing the magic method from the first recognizable class set, and classifying the class subset into a vulnerability detection result.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of object serialization and deserialization, and more particularly, to a method and apparatus for performing vulnerability detection on dynamic type objects. Background Art

[0002] Dynamic programming languages, such as Python, PHP, JavaScript, etc., show greater flexibility than static programming languages ​​such as C, C++, Java, etc. Dynamic programming languages ​​allow automatic inference and change of object types, addition and modification of methods and properties, etc. at runtime.

[0003] With the rapid development of artificial intelligence and the continuous evolution of programming paradigms, dynamic programming languages ​​have gradually become the basic development languages ​​in the fields of artificial intelligence, large models, the Internet of Things, etc., thanks to their advantages such as high development efficiency, ease of use, and cross-platform, and are favored by the majority of developers.

[0004] Dynamic typing is a unique programming concept in dynamic programming languages ​​and is also a core component of dynamic programming languages. Dynamic type objects do not need to be strongly bound to types when they are defined. The program can make judgments based on the behavior of dynamic type objects at runtime, determine the specific type of the object, and call corresponding functions and variables to complete the operation. This design can enhance the adaptability and scalability of the code.

[0005] In dynamic language programming, developers often use serialization technology to persist objects in various files to facilitate object transmission and sharing. When in use, the byte stream stored in the file is restored to the object through deserialization, and its functions and properties are called.

[0006] The deserialization operation of dynamic programming languages ​​can restore various types of deserialized data to objects without knowing the specific type of the object data in advance, and dynamically bind function methods and variable properties according to the object type expected by the developer when restoring the data. Combined with the non-strong type binding characteristics of dynamic types, it brings more possibilities for deserialization operations, but also introduces the problem of difficulty in type inference during deserialization for developers, which becomes a potential attack vulnerability. If used improperly, dynamic type objects will be deserialized into object types that are not expected by developers, thereby executing incorrect function methods or calling incorrect variable properties.

[0007] Therefore, we hope to have a solution that can detect the above vulnerabilities in massive codes through technical means, provide reference for developers, detect and improve the code, and improve robustness. Summary of the invention

[0008] One or more embodiments of this specification describe a method and device for vulnerability detection for dynamic type objects, which can analyze the deserialization operation of dynamic type objects and provide potential vulnerabilities that may cause attacks to solve the technical problems mentioned above.

[0009] According to a first aspect, a method for vulnerability detection for a dynamic type object, wherein the dynamic type object is generated by a target deserialization operation, the method comprising:

[0010] Obtain a target code file, which contains a target function for performing the target deserialization operation.

[0011] Taking the target function as a starting point, the corresponding code file is searched back along the sequential calling relationship of the function to obtain a plurality of call chains, including the first call chain.

[0012] For any first code file in the first call chain, a corresponding first reference class set is determined according to the first program element and the first reference information therein, where the first reference class set includes a class that defines the first program element.

[0013] Based on the collection of reference class sets corresponding to each code file on the first call chain, a first identifiable class set is obtained, and a class subset containing magic methods is determined from the first identifiable class set and included in the result of the vulnerability detection.

[0014] In one implementation, the first program element includes: a function and a variable.

[0015] In one implementation, determining the corresponding first reference class set includes:

[0016] The first program element is used as the target program element and the first reference information is used as the target reference information, target expansion is performed, and the target reference class set obtained by the target expansion is classified into the first reference class set. The target expansion includes:

[0017] Classifying the classes containing the target program element definition in the current code file and the classes referenced in the target reference information into a target reference class set;

[0018] For each referenced item declared in the target reference information, a search is performed according to the type of the referenced item, and the obtained reference class set is classified into the target reference class set.

[0019] In one scenario of the above implementation, the referenced item includes a first function, and the expanded search includes:

[0020] A first class including the first function definition is determined and classified into the target reference class set.

[0021] According to one implementation of this scenario, the first class is not defined in the first code file.

[0022] In a scenario of the above implementation, the referenced item includes an external class, and the external class is not defined in the first code file, and the expansion search includes:

[0023] determining a second code file containing the external class definition;

[0024] The target expansion is performed by taking the second program element of the second code file as the target program element and the second reference information as the target reference information.

[0025] In one scenario of the above implementation, the referenced item includes a target module, and the expanded search includes:

[0026] Obtain a third code file corresponding to the target module;

[0027] The target expansion is performed by taking the third program element of the third code file as the target program element and the third reference information as the target reference information.

[0028] In one scenario of the above implementation, the referenced item includes a target library, and the expanded search includes:

[0029] Obtaining the source file package of the target library;

[0030] For any fourth code file in the source file package, the fourth program element therein is used as the target program element, and the fourth reference information is used as the target reference information to perform the target expansion.

[0031] In one implementation, determining the class subset containing the magic method further includes:

[0032] Based on any target class in the class subset, subclasses having an inheritance relationship with the target class are classified into the class subset.

[0033] In one implementation, the target deserialization operation includes one or more of the following:

[0034] pickle.load, pickle.loads, yaml.load, cPickle.load.

[0035] According to a second aspect, a device for performing vulnerability detection on a dynamic type object, wherein the dynamic type object is generated by a target deserialization operation, the device comprising:

[0036] The acquisition module is configured to acquire a target code file, which contains a target function for performing the target deserialization operation.

[0037] The call chain generation module is configured to, starting from the target function, trace back along the sequential calling relationship of the function to search for the corresponding code file, and obtain a plurality of call chains, including the first call chain.

[0038] The reference determination module is configured to determine, for any first code file in the first call chain, a corresponding first reference class set based on the first program element and the first reference information therein, wherein the first reference class set includes a class that defines the first program element.

[0039] The vulnerability determination module is configured to obtain a first identifiable class set based on a collection of reference class sets corresponding to each code file on the first call chain, determine a class subset containing magic methods from the first identifiable class set, and include it in the result of the vulnerability detection.

[0040] According to a third aspect, a computer program product is provided, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0041] According to a fourth aspect, a computing device is provided, comprising a memory and a processor, wherein an executable code is stored in the memory, and when the processor executes the executable code, the method described in the first aspect is implemented.

[0042] In the method and device provided in the embodiments of this specification, the function that calls the deserialization method is targeted, and based on the sequential calling relationship of the function, the code file containing the definition of the calling function is backtracked and linked in sequence to form a call chain. By analyzing the dependency relationship (i.e., reference relationship) of each code file on the call chain, the class that can be identified by the dynamic type object targeted by the deserialization method is determined, and the class containing the magic method definition can be determined as a potential vulnerability point. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1A Provides an example deserialization vulnerability code;

[0045] Figure 1BProvides an exemplary deserialization vulnerability attack scenario;

[0046] Figure 1C Provides an example dynamic type deserialization vulnerability code;

[0047] Figure 2 This is a scenario of vulnerability detection for dynamic type objects disclosed in the embodiments of this specification;

[0048] Figure 3A An exemplary import relation syntax tree diagram is provided;

[0049] Figure 3B An exemplary inheritance relationship syntax tree diagram is provided;

[0050] Figure 3C An exemplary definition-use relationship syntax tree diagram is provided;

[0051] Figure 3D An exemplary parent-child relationship syntax tree diagram is provided;

[0052] Figure 3E An exemplary call relationship syntax tree diagram is provided;

[0053] Figure 4A A flow chart of a method for vulnerability detection for dynamic type objects is provided;

[0054] Figure 4B A flow chart of the method of target development is provided;

[0055] Figure 5 A schematic diagram of a class relationship diagram provided according to an embodiment of this specification;

[0056] Figure 6 A schematic diagram of a corresponding scenario for expanded retrieval provided according to an embodiment of this specification;

[0057] Figure 7 A schematic diagram of a device for performing vulnerability detection on dynamic type objects according to an embodiment of the present specification. DETAILED DESCRIPTION

[0058] The following describes the solution provided by the embodiments of this specification in conjunction with the accompanying drawings. It should be understood that the embodiments of this specification generally take Python language as an example, but it does not limit the application scenarios of the embodiments. In other scenarios, it can also be applied to other dynamic programming languages ​​that support dynamic types.

[0059] As mentioned above, serialization and deserialization technologies are crucial in software engineering. They form the basis for data exchange, object storage, and remote calls between systems. Serialization is the process of converting objects in a program into bytes-like objects that can be stored or transmitted. The converted objects are called serialized objects, which can make the objects persist beyond the life cycle of the program. In contrast, deserialization is the process of restoring serialized objects to objects that can be operated in memory. The restored objects are called deserialized objects. In practical applications, different programming languages ​​have developed different serialization / deserialization technologies based on their own characteristics.

[0060] Dynamic programming languages ​​usually provide more flexible and concise serialization / deserialization mechanisms to meet the needs of developers, and the combination of deserialization and dynamic types is particularly prominent. In dynamic language programming, developers can deserialize serialized objects without prior knowledge of specific data types, restore them to operable deserialized objects, and then determine their types based on the specific operations of the deserialized objects (including but not limited to function calls and variable calls), and dynamically bind function methods and variable properties.

[0061] However, any mechanism has its two sides. The dynamic type feature provides developers with a more flexible and lightweight programming experience in deserialization operations, but also provides attackers with attack vulnerabilities. The inventor found that the attacker can forge the byte stream in the deserialization operation so that the deserialized object is restored to a type that the developer did not expect. In the definition of the unexpected type, the attacker can forge a function method (for example, overload, rewrite) so that the deserialized object executes the attack code contained therein, thereby attacking the system. The characteristics of the dynamic type magnify such vulnerabilities in the program, because the deserialized object can be restored to a type that can be recognized by any program, that is, the more classes referenced in the program, the more possible types of types that the deserialized object may be instantiated. Therefore, the abuse of dynamic types will expand the scope of attack vulnerabilities in deserialization operations.

[0062] The inventors further discovered that attacks on deserialization using magic methods are particularly covert and difficult to be discovered by developers. This is because magic methods are automatically called by the program at a specific time in a special way (implicit call), and can override some built-in behaviors of the object by overwriting. Magic methods are used to achieve polymorphism, that is, objects of different data types can have the same method name, but their respective method implementations are different.

[0063] Taking Python as an example, magic methods start and end with double underscores (__), such as the __reduce__ function. Magic methods are an important part of implementing dynamic typing (Duck Typing) in Python, allowing developers to customize the behavior of objects without modifying the objects themselves. This also leaves a handle for attackers to manipulate. Attackers can rewrite magic methods that are implicitly called in specific object operations in the target data type and embed attack code, so that when the program executes specific object operations, the attack code is automatically executed to achieve the attack purpose. For example, the magic method __init__ is automatically called when a new object is created. When the deserialized object is restored, this method is automatically called. Attackers can use this method to attack the system. The magic method __reduce__ is implicitly called at the end of deserialization and returns a tuple, which can contain a callback function and its various call parameters. Attackers can manipulate this method to make the program execute various attack commands (even form RCE vulnerabilities, i.e. Remote Code Execution), forming an attack on the system. The magic method __iter__ is implicitly called when iterating over the elements in the deserialized object. Attackers can also attack the system by rewriting this method. There are a large number of magic methods in Python, which can be called by the system when a specific operation occurs. I will not list them one by one here.

[0064] Next, we will combine examples to further describe the attack vulnerabilities that may be formed during the deserialization process of the above dynamic types. In the example, Python language will be used as an example. The module pickle implements the serialization and deserialization of a Python object structure. Among them, pickle.dumps(object) is used to serialize an object, object is the target object to be serialized, and the function will return the serialized byte stream. pickle.loads(data) is used to deserialize and restore the byte stream to an object, data is the target byte stream. Similar deserialization functions include pickle.load(file) for reading the byte stream in the specified file (file) and deserializing it into an object. yaml.load(ymlFile) is used to deserialize file data (ymlFile) stored in YAML format into an object. And the more efficient deserialization function cPickle.load() provided by the deserialization library written in C language. When using these deserialization functions, if the serialized object / byte stream in the input parameter is untrusted and unverified, the embedded malicious code and unauthorized system behavior may be executed during the deserialization process, causing serious security issues.

[0065] Figure 1A An example deserialization vulnerability code is shown, which is used to illustrate the implicit call of a magic method during the deserialization process. Figure 1A , the Bad class is defined in lines 3-5 of the program, which contains the definition of the magic method __reduce__, which will be called by the program itself when the Bad type object is deserialized. Object b is defined in line 7 of the program, instantiated as a Bad type object, and serialized into a byte stream bad_token through the serialization function "pickle.dumps" in line 8. In some specific scenarios, bad_token can be used for persistence or variable exchange outside the program. The program holding bad_token can restore it to an object through the deserialization function "pickle.loads". As shown in line 9 of the example, bad_token is restored to the object new_b through the deserialization operation. During this operation, the program automatically detects that the object new_b should be of Bad type, so the magic method __reduce__ in the Bad class will be implicitly called. If the magic method is maliciously manipulated by an attacker and malicious code is embedded, the malicious code will also be executed. For example, the malicious code shown in line 5 of the example will execute the "whoami" command in the target machine system by calling back the os.system() function when the __reduce__ function is executed, thus attacking the target machine. Figure 1B An exemplary deserialization vulnerability attack scenario is shown, in which the attacker manipulates the magic method in the Bad class and embeds the attack code "NETSTAT-nao". When the target system performs a deserialization operation on the byte stream, the attack code will be automatically executed to read out all open network ports in the target system.

[0066] Based on the above sample code, further combining it with the application of dynamic types will expand the scope of attack vulnerabilities that exist in the deserialization operation process. Figure 1C An exemplary dynamic type deserialization vulnerability code is shown, which is used to illustrate a method of attacking by using type confusion in the deserialization process. Figure 1C, the Bad class is defined in lines 3-5 of the program, which contains the definition of the magic method __reduce__ embedded with malicious code. The Duck class is defined in lines 7-9 of the program, which contains the function look, which will output the string "Duck!". In line 11 of the program, the byte stream "userinput" is restored through the deserialization function to obtain the object d. Combined with line 12, the look function of object d is called, which shows that the developer expects d to be of type Duck and output the string "Duck!". However, attackers can achieve the purpose of attacking the system by manipulating the byte stream in the deserialization function. This is because dynamic programming languages ​​do not strongly bind types to deserialized objects when deserializing, that is, type enforcement is not performed when the program is compiled, but the type of the deserialized object is dynamically determined based on its behavior when the program is running. Therefore, once the attacker forges a byte stream (userinput), for example, a serialized byte stream of a Bad type object, the object d will be deserialized into a type that the programmer does not expect. Then, when the program executes to line 11, the magic method __reduce__ in the Bad class has been implicitly called, completing the attack on the system. In other words, since dynamic types can dynamically determine their own types according to the specific function methods or variables called when used, the application of dynamic types means that all types that may be identified as deserialized objects have vulnerabilities that may be attacked.

[0067] From the above cases, we can see that the attack vulnerabilities exposed in the deserialization operation are caused by the abuse of dynamic type characteristics. In other words, the dynamic type mechanism allows the type of an object to be determined based on the function method and variable attribute currently called. Therefore, when a deserialized object calls different function methods / variable attributes, it can be displayed / instantiated as a corresponding new type, which may be an unexpected type for the developer and contains attack code. In addition, when a deserialized object calls a function method with multiple accessible definitions (for example, method overriding, virtual method, inheritance relationship, etc.), even if there is a unique called method determined by the method resolution order (for example, the MRO mechanism in Python, Method Resolution Order), all classes containing the accessible definition of the function method will be automatically loaded by the program, thereby expanding the range of classes that can be identified by the deserialized object.

[0068] To intuitively explain the above findings, assume that there is a deserialization operation in a piece of program code A, and the range of classes recognizable by this deserialization operation is defined as R(A). The function methods that can be accessed by directly referencing classes in A are defined as R1, and the corresponding range of referenced classes is R1(A); the function methods that can be accessed by indirectly referencing classes in them by referencing modules (for example, referencing *.py code files) are defined as R2, the set of referenced modules is represented as M(A), and the range of referenced classes extracted from the module set is R2(M(A)). These referenced classes may contain magic methods. It should be noted that among the classes included in R1(A) and R2(M(A)), there may be definitions of function methods with the same name, but the class names cannot be repeated (otherwise, problems such as Name Error, Import Error, or definition overwrite may occur), that is, the intersection of R1(A) and R2(M(A)) is empty.

[0069] In a scenario, assume that the deserialized object calls two different methods b and c, which are defined in two different classes respectively, and the classes containing methods b and c can be referenced directly or indirectly. Assume that the class B containing the definition of method b is directly referenced, denoted as B∈R1(A), and the class containing the definition of method c is indirectly referenced by referencing module C, denoted as R2(C)∈R2(M(A)). In this case, if the dynamic type mechanism is used to simultaneously call methods b and c using this deserialized object, then the range of types that the deserialized object may be recognized as is R(A) = R1(A) + R2(M(A)). If the dynamic type mechanism is not used and methods b and c are called in different code spaces respectively, then it is only necessary to reference the classes containing the definitions of the called methods in the corresponding code spaces. At this time, the range of types that the deserialized object may be recognized as is R'(A) = R1(A) - B + R2(M(A)) (when calling method c), or R”(A) = R1(A) - R2(C) + R2(M(A)) (when calling method b). Since generally, the range of classes R2(C) introduced by referencing modules is larger than the range of direct class introduction B, it can be inferred that R”(A) < R'(A) < R(A). The larger the range of referenced classes, the higher the probability of containing attackable magic methods. Thus, it can be seen that applying dynamic types may increase the risk of being attacked in deserialization operations.

[0070] In another scenario, assume that the deserialized object calls method d, which is defined in two different classes D1 and D2. The two classes can be referenced directly or indirectly. Assume that class D1 is directly referenced, denoted as D1 ∈ R1(A), and class D2 is indirectly referenced through reference module D, denoted as R2(D) ∈ R2(M(A)). In this case, when method d is called, due to the existence of the dynamic type mechanism, the classes containing the definition of method d may all be the target types into which the deserialized object is instantiated. Then the range of types that the deserialized object may be recognized as is R(A) = R1(A) + R2(M(A)). And if the definition of method d is unique, that is, method d is either defined in D1 or defined in D2, at this time the range of types that the deserialized object may be recognized as is R'(A) = R1(A) - D1 + R2(M(A)) (when d is defined in D2), or R”(A) = R1(A) - R2(D) + R2(M(A)) (when d is defined in D1). Similar to the analysis in the previous scenario, it can be inferred that R”(A) < R'(A) < R(A), and thus it can also be known that applying dynamic types may increase the risk of being attacked in deserialization operations.

[0071] Through the above description, it can be understood that in the deserialization operation of dynamic programming languages, applying or misusing dynamic types may potentially introduce attack vulnerabilities in the program. Due to the non-strong type checking feature of dynamic programming languages, these attack vulnerabilities are not easily detected by developers during program writing or compilation. Therefore, it is necessary to use technical means to discover these potential attack vulnerabilities, so as to actively adjust the code and prevent the vulnerabilities from being exploited during program operation to form malicious attacks.

[0072] Based on the above technical findings, the inventors specifically proposed corresponding solutions. In the embodiments of this specification, a method and device for detecting vulnerabilities in dynamic type objects are proposed, which can detect potential deserialization attack vulnerabilities (hereinafter also referred to as the attack surface) of dynamic type objects in the program. Further elaboration will be given below in combination with the embodiments and the accompanying drawings.

[0073] As mentioned above, attacks against deserialization of dynamic type objects usually occur in the classes recognizable by the deserialization operation. Therefore, the range of classes recognizable by the deserialization operation can be determined by enumerating the reference classes related to the deserialization operation, and then the classes containing magic methods are determined among them, which are the potential attack surfaces.

[0074] In Python, classes can be referenced in multiple ways:

[0075] Direct definition: Directly declare the definition of the class in the current code file and use it within the code file.

[0076] Direct import: Use the import statement to import a specific class into the current code.

[0077] Indirect import: Use the import declaration to directly import all the contents of a module into the current code, thereby indirectly causing all classes in the module and its dependent reference classes to be imported as well.

[0078] Built-in classes: Classes defined in commonly used underlying system packages, usually automatically referenced by programs, for example, code libraries configured in system environment variables.

[0079] Therefore, when determining the scope of classes that can be identified by the deserialization operation, it should not be limited to the classes in the code file where the deserialization operation occurs, but also consider the calling relationship of the deserialization operation and the code file where the calling function is located. In addition, due to Python's import mechanism, when an external class (the definition of the class is not in the current file) is referenced, all import information contained in the code file that defines the external class will also be imported at the same time to ensure that all dependencies of the external class are automatically loaded. Therefore, the referenced classes in the import information contained in such code files must also be included in the types that can be identified by the deserialization operation. Similarly, when an external library (External Library, such as the numpy library) is referenced, the referenced classes imported by all code files contained in the external library must also be included in the types that can be identified by the deserialization operation.

[0080] Figure 2 A scenario of vulnerability detection for dynamic type objects disclosed in the embodiment of this specification is shown. First, the code file of the program is analyzed, and several relationships contained in the code file can be obtained, including: import relationship, inheritance relationship, definition-use relationship, parent-child relationship and call relationship. The analysis and extraction methods of various relationships in the program will be specifically explained below. Then, taking the deserialization operation in the program as the origin, a class relationship diagram is constructed. The diagram takes the code file as the basic component unit, and the code files are linked to form a call chain, indicating that the functions related to the deserialization operation are called in sequence. In addition, each code file in the class relationship diagram also contains the reference class information on which each depends. Finally, by analyzing the reference class set of all code files on the call chain (indicating the class range that the deserialization operation can identify), the attack vulnerability that the deserialization operation in the program may contain, that is, the attack surface, is determined.

[0081] The several relationships contained in the code files described above can be analyzed by various technical means. In one embodiment, keyword analysis can be performed on the source code files. For example, the import keyword corresponds to the import relationship, the def keyword corresponds to the definition relationship, and so on.

[0082] In another embodiment, various relationships contained in the program code can be obtained by analyzing the Abstract Syntax Tree (AST). AST is a tree-like structure in which each node represents a grammatical structure in the program code, such as a module, a class declaration, a function definition or a function call, etc. Some parsing examples of grammatical relationships are given below to help illustrate how to extract various relationships in a program code file through AST.

[0083] Import relationship: records the modules, functions, and classes referenced in the code file. Figure 3A An example of an import relationship syntax tree is given. In the left branch of the exemplary syntax tree, the reference to sys in the code file can be obtained through the import node of AST (corresponding to the source code "import sys"); in the right branch of the exemplary syntax tree, the reference to import_function in the module pyimports in the code file can be obtained through the import_from node of AST (corresponding to the source code "from pyimports import import_function"). It should be noted that the types of reference items sys and import_function (module, library, function or class) cited in the example cannot be determined at present, and need to be combined with other relationships (for example, definition relationships) for judgment. In addition, in Python, there are many different import forms, such as: "import x as y", "from xy import k", etc., which can be extracted by traversing the import and import_from branches in AST, and examples are not given one by one here.

[0084] Inheritance relationship: records the inheritance relationship between classes in the code file. Figure 3BAn example of an inheritance relationship syntax tree is given. In this exemplary syntax tree, the branch containing the definition of classC is marked with a dotted frame, where class_def indicates that this node and its child nodes contain the definition of the class, the node pointed to by name indicates that this branch is the class definition of classC, the branch pointed to by bases indicates that this branch is the definition of the inherited base class of classC, and [list] contains all the base classes of classC, classA and classB. This can parse the relationship between class definition and class inheritance in the code file.

[0085] Definition-Use Relationship: Records the definition and use of classes, functions, and variables. Figure 3C An example of a definition-use relationship syntax tree is given. In this exemplary syntax tree, the left dotted box marks the branch containing the definition of the function method FuncA, where function_def indicates that this node and its child nodes contain the definition of the function method, the node pointed to by name indicates that this branch is the definition of the FuncA function method, and the arguments node contains the input parameters used in the function method. The right dotted box marks the definition and use of the variable a, where the assign node indicates that this node and its child nodes contain the definition and use of the variable a (assignment in this example). The definition-use relationship contained in the code file can thus be parsed from the AST. It should be understood that the above examples do not represent an exhaustive list of use cases for the definition-use relationship analysis. In actual applications, a variety of definitions and use relationships are contained in the AST. For example, the use relationship of loop calls can be extracted by parsing the for node, and the definition of static variables can also be obtained by parsing the constant node, etc. Examples are not given one by one in the embodiments of this specification.

[0086] Parent-child relationship: record the function method definition in the class, and the variable attribute definition in the function method. Figure 3D An example of a syntax tree of a parent-child relationship is given. In this exemplary syntax tree, the class_def node indicates that the child nodes contained in the node are all definitions of functions and attributes in the class classA, where the assign node contains the definition of the variable attribute self in the classA class, and the function_def node contains the definition of the function method Intro in the classA class. From this, the parent-child relationship contained in the code file can be analyzed. It can be seen that by combining the comprehensive judgment of the parent-child relationship, the definition relationship and the import relationship, the type of the referenced item in the reference declaration can be known.

[0087] Call relationship: record the calls of function methods and variable attributes in the code. Figure 3EAn example of a call relationship syntax tree is given. In the exemplary syntax tree, the assign node marked with a dotted box 1 indicates that the branch contained in the node and its child nodes is an instantiation of the classA type object instanceA. The expr node marked with a dotted box 2 indicates that the branch contained in the node and its child nodes is a call to the function method Intro of the object instanceA (i.e. instanceA.Intro()). The assign node marked with a dotted box 3 indicates that the branch contained in the node and its child nodes is a call to the variable attribute a of the object instanceA (i.e. instanceA.a=2). In this way, the call relationship in the code file can be analyzed from the AST.

[0088] The above is an overview of the technical ideas of the exemplary embodiments of this specification. Figure 4A In the figure, a flow chart of a method for vulnerability detection for dynamic type objects provided according to an embodiment of this specification is shown. It can be understood that the method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities. The following will first introduce the main process of the method in conjunction with the embodiment, and then further expand the preferred method of the key steps therein.

[0089] See also Figure 4A In one embodiment, the dynamic type object is generated by a target deserialization operation, and the method includes at least the following steps: Step S401: Obtain a target code file, which contains a target function for performing the target deserialization operation. Step S403: Starting from the target function, trace back and search for the corresponding code file along the sequential calling relationship of the function to obtain a number of call chains, including a first call chain. Step S405: For any first code file in the first call chain, determine the corresponding first reference class set according to the first program element and the first reference information therein, and the first reference class set contains the class that defines the first program element. Step S407: Based on the collection of reference class sets corresponding to each code file on the first call chain, obtain a first identifiable class set, determine a class subset containing magic methods from the first identifiable class set, and include it in the result of the vulnerability detection.

[0090] The method for performing vulnerability detection on dynamic type objects will be explained in detail below in conjunction with an embodiment.

[0091] In step S401, a target code file is obtained, which contains a target function for performing the target deserialization operation. For several code files in a program, the target function containing the deserialization operation can be located by analyzing its abstract syntax tree, and the code file containing the target function can be determined as the target code file.

[0092] In a specific scenario, the target function is located by traversing and searching the call node under the function definition in the AST and determining whether the called function belongs to a deserialization operation function. If it belongs to a deserialization operation, the code file containing the target function definition can be determined as the target code file.

[0093] According to one implementation, the target deserialization operation may include: pickle.load, pickle.loads, yaml.load, cPickle.load. The meanings of these deserialization operation functions have been introduced above and will not be repeated here.

[0094] Next, taking the target function located in any target code file as an example, in step S403, starting from the target function, the corresponding code file is searched back along the sequential calling relationship of the function to obtain several call chains, including the first call chain. In this step, by parsing the function call relationship, a directed graph with code files as basic components and function call sequence as the link direction can be obtained, which is referred to as a class relationship graph (CRG) in the embodiment of this specification. In other words, CRG starts from the code file containing the target function, and traces back according to the function call relationship, searching for calling functions in sequence until a function that has not been called is retrieved; then, according to the call relationship and the code file to which each calling function belongs, these code files are linked together, and the obtained several call chains constitute CRG.

[0095] Figure 5 An exemplary class relationship diagram is shown. In this CRG, the function foo contained in the code file f.py calls the pickle.load() deserialization operation. foo is the target function and f.py is the target code file. According to AST analysis, it can be known that in the code file c.py, the function coo and the function boo both directly or indirectly call the function foo. Therefore, the f.py and c.py code files are linked. Similarly, c.py and a.py can be linked because the function aoo defined in a.py is the calling function of the function boo; c.py and d.py can also be linked because the function doo defined in d.py is the calling function of the function coo. Thus, two call chains are formed in the CRG: f.py→c.py→a.py and f.py→c.py→d.py. The code file e.py containing the function eoo that calls the deserialization operation yaml.load() can be processed in the same way to obtain another call chain e.py→d.py.

[0096] Any call chain in CRG contains various code files related to a specific deserialization operation, as well as the reference information that each code file depends on. Next, based on any call chain in CRG, the reference information contained on it will be analyzed to extract the classes that can be identified by the deserialization operation.

[0097] Specifically, in step S405, for any first code file in the first call chain, a corresponding first reference class set is determined according to the first program element and the first reference information therein, where the first reference class set includes a class that defines the first program element.

[0098] Take a code file on the call chain as an example. The code file (usually in the header) contains the code resources that the code file depends on, which is represented by reference information consisting of several referenced items (import declarations). Combined with the introduction to the characteristics of deserialization and dynamic types in the previous article, it can be known that the referenced class is the class that the target deserialization operation may recognize.

[0099] Therefore, in this step, based on the import relationship and parent-child relationship extracted from the AST, the reference class contained in the first reference information in the first code file is determined and included in the first reference class set. At the same time, considering that some classes are defined and used in this file, these classes will not be explicitly referenced as referenced items in the header of the code file. Therefore, these classes directly defined in the first code file are included in the first reference class set. In other words, according to the definition-use relationship extracted from the AST, several program elements are retrieved in the first code file, and then according to the parent-child relationship, the classes containing the definitions of the program elements are determined, and these classes are included in the first reference class set.

[0100] The above program elements may be atomic attributes in a code file. In some embodiments, the first program elements include: functions and variables.

[0101] Furthermore, based on the introduction of class reference methods in the previous article and the explanation of the deserialization operation type identification mechanism in dynamic programming languages, when analyzing the types that can be identified by the target deserialization operation, not only the referenced classes contained in the current file reference information should be included, but also other types of referenced items should be further analyzed to ensure that all classes that may be identified by the target deserialization operation are retrieved. The specific process of analyzing other types of referenced items will be explained in detail below. Here, the above brief introduction to the retrieval strategy is given first.

[0102] In step S407, based on the collection of reference class sets corresponding to each code file on the first call chain, a first identifiable class set is obtained, and a class subset containing magic methods is determined from the first identifiable class set and included in the result of the vulnerability detection.

[0103] In this step, the reference class sets corresponding to each code file on any call chain are summarized to obtain all possible classes that the target deserialization operation can identify in the call chain. According to the analysis of the causes of the attack vulnerability in the previous article, classes with magic methods will be the targets of malicious code injection and type confusion by attackers. Therefore, by analyzing the definition-use relationship and parent-child relationship in AST, classes containing magic methods are found from the first identifiable class set. The class subset composed of these classes is the attack surface for abusing dynamic types to perform target deserialization operations.

[0104] In some embodiments, considering that the subclass will obtain the function methods of the parent class / base class (including the magic methods of the parent class) by default, the attack surface includes not only the target class containing the magic method, but also the subclasses of these target classes. Therefore, in order to obtain more accurate and comprehensive vulnerability detection results, the inheritance relationship obtained from the AST can be analyzed, and based on any target class in the class subset, the subclasses with inheritance relationship with the target class are classified into the class subset.

[0105] The above is an introduction to the main process of a method for vulnerability detection for dynamic type objects provided in an embodiment of this specification. It should be understood that the above steps S401-S407 can be repeatedly executed. Through multiple rounds of analysis and retrieval of different target code files and various call chains, the attack vulnerabilities that may be introduced by several deserialization operations on dynamic type objects contained in the program are comprehensively retrieved, thereby obtaining a complete attack surface analysis result to provide a reference for developers.

[0106] In some embodiments of the present specification, a method for searching different types of referenced items contained in reference information in a code file is also provided to ensure the accuracy and comprehensiveness of reference-based search results based on CRG. Figure 4B The process diagram of the search method is shown. Taking the aforementioned step S405 as an example to expand the method, the determination of the first reference class set in the step includes step S4051: taking the first program element as the target program element and the first reference information as the target reference information, and performing target expansion.

[0107] The target is expanded to recursively search for each reference item in the code file, wherein in a single round, the classes directly defined in the current code file and the classes directly referenced in the reference information of the code file are all taken as target reference classes and classified into the target reference class set. Different search processes are performed on each reference declaration in the reference information of the code file according to the type of the referenced item to obtain the reference class set that the referenced item depends on, which is also classified into the target reference class set.

[0108] After the above target expansion is completed, the target reference class set obtained by the target expansion is classified into the first reference class set.

[0109] As a preferred solution of step S405, according to one embodiment, the target expansion is started with any first code file in step S405 as the current code file.

[0110] In step S4053, the classes containing program element definitions in the current code file (ie, directly defined classes) and the classes directly referenced in the reference information are classified into a target reference class set.

[0111] Next, in step S4055, each referenced item declared in the reference information of the current code file is searched according to the type of the referenced item, and the obtained reference class set is classified into the target reference class set. In this step, the type of the referenced item may be a class, a function, a module or a library. Different types of referenced items can be processed using different search methods to obtain the reference class set that the referenced item depends on and classify it into the target reference class set. Figure 6 A schematic diagram is provided to illustrate the scenarios corresponding to different types of cited items in expanded retrieval.

[0112] According to one implementation, the referenced item includes a first function, and the expanded search includes: determining a first class including the definition of the first function and classifying the first class into the target reference class set. Figure 6 The referenced item indicated by the arrow marked as 1 includes a reference to the method Intro. According to the definition-usage relationship and parent-child relationship extracted in the AST analysis, it can be determined that the method Intro is defined in class classA. Therefore, classA (i.e., the first class) can be included in the target reference class set.

[0113] In one scenario of the above implementation, the first function may be a function defined in an external code file, that is, the first class is not defined in the first code file. At this time, the class defining the first function may also be included in the target reference class set.

[0114] According to one implementation, the referenced item includes an external class that is not defined in the first code file (current code file), and the expansion search includes: determining a second code file that includes the definition of the external class. Using the second program element of the second code file as the target program element and the second reference information as the target reference information, the target expansion is performed. Figure 6 The referenced item indicated by the arrow marked as 2 in the figure includes a reference to class classB. According to the definition-use relationship and parent-child relationship extracted in the AST analysis, it can be known that classB is not defined in the current code file, but in the moduleB.py code file (the second code file). At this time, the program element contained in the second code file can be used as the target program element, the reference information contained in the second code file can be used as the target reference information, and the second code file can be used as the current code file for target expansion.

[0115] According to one implementation, the referenced item includes a target module, and the expansion search includes: obtaining a third code file corresponding to the target module. Using the third program element of the third code file as the target program element and the third reference information as the target reference information, the target expansion is performed. Figure 6 The referenced items indicated by the arrow marked as 3 in the figure include references to module moduleC (usually, in Python, a module generally refers to a py file containing Python code). At this time, the program elements contained in moduleC.py (the third code file) can be used as target program elements, the reference information contained in it can be used as target reference information, and the third code file can be used as the current code file for target expansion.

[0116] According to one implementation, the referenced item includes a target library, and the expansion search includes: obtaining a source file package of the target library. For any fourth code file in the source file package, the fourth program element therein is used as the target program element, and the fourth reference information is used as the target reference information to perform the target expansion. Figure 6 The referenced items indicated by the arrows marked as 4 in the figure include references to the library libA (usually, in Python, a library generally refers to a series of py files organized together for the purpose of achieving a certain function). At this time, you can first obtain the source files contained in the library (for example, the py files A1, A2, B1, B2 contained in libA, which can be obtained through "pip install libA"), take the program elements contained in each source file (the fourth code file) as the target program elements, the reference information contained as the target reference information, take the fourth code file as the current code file, and perform target expansion.

[0117] The above is a detailed explanation of the target expansion operation. After the target expansion of the reference information of the first code file is completed, as a conclusion to step S405, in step S4057, the target reference class set obtained by the target expansion is classified into the first reference class set, and the reference class set corresponding to the first code file can be obtained.

[0118] According to one or more embodiments, a method for vulnerability detection for dynamic type objects is described in detail above. By adopting the above method provided in the embodiments of this specification, by performing AST analysis on the code files in the program, building a class relationship diagram, and analyzing the referenced items, the deserialization operations of the dynamic type objects contained therein can be comprehensively analyzed to identify potential attack vulnerabilities, provide references for developers, detect and improve the code, and improve the robustness of the program.

[0119] In this specification, the word "first" in terms such as the first code file and the first reference information, as well as the corresponding "second", "third" (if any), etc. in the text are merely for the convenience of distinction and description and do not have any limiting meaning.

[0120] The foregoing describes certain embodiments of the present specification, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in an order different from that in the embodiments, and the desired results may still be achieved. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0121] Figure 7 Schematic diagram of a device for vulnerability detection for dynamic type objects according to an embodiment of this specification. The device 700 is deployed in a computing device, and the computing device can be implemented by any device, equipment, platform, device cluster, etc. with computing and processing capabilities. Figure 4A The method embodiment shown corresponds to the method embodiment shown in FIG. 1 . The dynamic type object is generated by a target deserialization operation, and the device 700 includes:

[0122] The acquisition module 701 is configured to acquire a target code file, which contains a target function for performing the target deserialization operation;

[0123] The call chain generation module 702 is configured to, starting from the target function, trace back and search the corresponding code file along the sequential call relationship of the function to obtain a plurality of call chains, including the first call chain;

[0124] The reference determination module 703 is configured to determine, for any first code file in the first call chain, a corresponding first reference class set according to the first program element and the first reference information therein, wherein the first reference class set includes a class defining the first program element;

[0125] The vulnerability determination module 704 is configured to obtain a first identifiable class set based on a collection of reference class sets corresponding to each code file on the first call chain, determine a class subset containing magic methods from the first identifiable class set, and include it in the result of the vulnerability detection.

[0126] According to another embodiment of the present specification, there is also provided a computer program product, including a computer program / instruction, which implements the aforementioned combination when executed by a processor. Figure 4A The steps of the method.

[0127] According to another embodiment, the present specification further provides a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the aforementioned combination is implemented. Figure 4A The steps of the method.

[0128] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present invention may be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions may be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0129] The specific implementation methods described above further describe the purpose, technical solutions and beneficial effects of the embodiments of the present invention in detail. It should be understood that the above description is only a specific implementation method of the embodiments of the present invention and is not intended to limit the scope of protection of the present invention. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solution of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A method for vulnerability detection for a dynamic type object, wherein the dynamic type object is generated by a target deserialization operation, the method comprising: Obtaining a target code file, which contains a target function for performing the target deserialization operation; Taking the target function as a starting point, searching the corresponding code file along the sequential calling relationship of the function to obtain a plurality of call chains, including a first call chain; For any first code file in the first call chain, determine a corresponding first reference class set according to the first program element and the first reference information therein, wherein the first reference class set includes a class defining the first program element; Based on the collection of reference class sets corresponding to each code file on the first call chain, a first identifiable class set is obtained, and a class subset containing magic methods is determined from the first identifiable class set and included in the result of the vulnerability detection.

2. The method according to claim 1, wherein: The first program elements include: functions and variables.

3. The method according to claim 1, wherein: The determining of the corresponding first reference class set includes: The first program element is used as the target program element and the first reference information is used as the target reference information, target expansion is performed, and the target reference class set obtained by the target expansion is classified into the first reference class set. The target expansion includes: Classifying the classes containing the target program element definition in the current code file and the classes referenced in the target reference information into a target reference class set; For each referenced item declared in the target reference information, a search is performed according to the type of the referenced item, and the obtained reference class set is classified into the target reference class set.

4. The method according to claim 3, wherein: The referenced item includes a first function, and the expanded search includes: A first class including the first function definition is determined and classified into the target reference class set.

5. The method according to claim 4, wherein: The first class is not defined in the first code file.

6. The method according to claim 3, wherein: The referenced item includes an external class, which is not defined in the first code file, and the expanded search includes: determining a second code file containing the external class definition; The target expansion is performed by taking the second program element of the second code file as the target program element and the second reference information as the target reference information.

7. The method according to claim 3, wherein: The referenced item includes a target module, and the expanded search includes: Obtain a third code file corresponding to the target module; The target expansion is performed by taking the third program element of the third code file as the target program element and the third reference information as the target reference information.

8. The method according to claim 3, wherein: The referenced item includes a target library, and the expanded search includes: Obtaining the source file package of the target library; For any fourth code file in the source file package, the fourth program element therein is used as the target program element, and the fourth reference information is used as the target reference information to perform the target expansion.

9. The method according to claim 1, wherein: Determining the subset of classes containing magic methods further includes: Based on any target class in the class subset, subclasses having an inheritance relationship with the target class are classified into the class subset.

10. The method according to claim 1, wherein: The target deserialization operation includes one or more of the following: pickle.load, pickle.loads, yaml.load, cPickle.load.

11. A device for performing vulnerability detection on a dynamic type object, wherein the dynamic type object is generated by a target deserialization operation, the device comprising: An acquisition module is configured to acquire a target code file, which contains a target function for performing the target deserialization operation; A call chain generation module is configured to, starting from the target function, trace back and search the corresponding code file along the sequential call relationship of the function to obtain a plurality of call chains, including a first call chain; a reference determination module configured to, for any first code file in the first call chain, determine a corresponding first reference class set according to the first program element and the first reference information therein, wherein the first reference class set includes a class defining the first program element; The vulnerability determination module is configured to obtain a first identifiable class set based on a collection of reference class sets corresponding to each code file on the first call chain, determine a class subset containing magic methods from the first identifiable class set, and include it in the result of the vulnerability detection.

12. A computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 10.

13. A computing device comprising a memory and a processor, characterized in that: The memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 10 is implemented.

Images