Method, device, storage medium and equipment for detecting python package dependency relationship

Abstract

The embodiment of the application provides a method, device, storage medium and equipment for detecting a Python package dependency relationship, in which, all possible dependent packages are found through module information of all imported modules in a to-be-analyzed package, and then it is determined whether a dependency relationship exists by comparing calling functions in the to-be-analyzed package and defined functions in each possible dependent package. In this way, the extracted dependency relationship covers both the "strong dependency" relationship of a dependency package declared in plaintext and the "weak dependency" relationship of a reference module without a declared dependency package, so that the accurate and comprehensive extraction of the dependency relationship between Python packages is realized, and the user can completely exclude the security risks caused by the attack on the dependent package.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field, in particular, to a method and device for detecting Python package dependency relationship, a storage medium and equipment. BACKGROUND

[0002] When a Python package declares its dependent packages in a setup.py file, installing the Python package will install its dependent packages synchronously. Such dependency can be referred to as "strong dependency". The existing Python package dependency graph is usually generated from "strong dependency". However, these dependency graphs are incomplete. When a package is attacked, only those packages directly dependent on the attacked package can be obtained through the existing dependency graph, which cannot completely eliminate the security risks caused by the dependent attacked package. SUMMARY

[0003] The embodiments of the present application aim to provide a method and device for detecting Python package dependency relationship, a storage medium and equipment, and solve the problem that the incomplete Python package dependency graph cannot completely eliminate the security risks caused by the dependent attacked package in the related art.

[0004] In a first aspect, the embodiments of the present application provide a method for detecting Python package dependency relationship, comprising:

[0005] Obtaining module information of all imported modules in a to-be-analyzed package;

[0006] Determining candidate dependent packages corresponding to the to-be-analyzed package according to the module information;

[0007] Obtaining calling functions in the to-be-analyzed package and defining functions in the candidate dependent packages; the defining functions are functions defined using the keyword def;

[0008] Determining whether there is a dependency relationship between the to-be-analyzed package and the candidate dependent packages according to the comparison result between the calling functions and the defining functions.

[0009] In the above implementation process, all possible dependent packages are found through the module information of all imported modules in the to-be-analyzed package, and then it is determined whether there is a dependency relationship by comparing the calling functions in the to-be-analyzed package and the defining functions in each possible dependent package. In this way, the extracted dependency relationship covers not only the "strong dependency" relationship of the explicitly declared dependent package, but also the "weak dependency" relationship of the referenced module without declaring the dependent package. Therefore, the accurate and comprehensive extraction of the dependency relationship between Python packages is achieved, and the user can completely eliminate the security risks caused by the dependent attacked package.

[0010] Further, in some embodiments, the module information comprises a package name corresponding to the import module; and the determining the candidate dependent package corresponding to the package to be analyzed according to the module information comprises:

[0011] searching a third-party package corresponding to the package name according to the package name corresponding to the import module;

[0012] determining the searched third-party package as the candidate dependent package corresponding to the package to be analyzed.

[0013] In the above implementation process, a solution for obtaining possible dependent packages of a package to be analyzed is provided, that is, a corresponding third-party package can be searched according to a package name of a third-party package corresponding to an import module, and the third-party package is used as a candidate dependent package of the package to be analyzed.

[0014] Further, in some embodiments, the calling function is obtained through an abstract syntax tree of the package to be analyzed; and the defining function is obtained through an abstract syntax tree of the candidate dependent package.

[0015] In the above implementation process, a solution for obtaining a calling function and a defining function is provided.

[0016] Further, in some embodiments, the defining function is obtained based on the following manner:

[0017] searching a FunctionDef node in an abstract syntax tree of the candidate dependent package, each node in the abstract syntax tree representing a structure in source code of the candidate dependent package;

[0018] determining the searched FunctionDef node as the defining function in the candidate dependent package.

[0019] In the above implementation process, a specific solution for obtaining a defining function is provided, that is, a FunctionDef node in an abstract syntax tree of a candidate dependent package is searched, and the FunctionDef node is determined as a defining function.

[0020] Further, in some embodiments, the calling function is obtained based on the following manner:

[0021] searching a Call node in each node in an abstract syntax tree of the package to be analyzed, each node in the abstract syntax tree representing a structure in source code of the package to be analyzed;

[0022] determining a function searched in the searched Call node as the calling function in the package to be analyzed.

[0023] In the implementation process, a specific solution for obtaining the calling function is provided, that is, searching for the Call node in each node in the abstract syntax tree of the package to be analyzed, and determining the function searched in the Call node as the calling function.

[0024] Further, in some embodiments, the determining whether the dependency relationship exists between the package to be analyzed and the candidate dependency package according to the comparison result between the calling function and the defining function comprises:

[0025] constructing a calling function set based on the calling function in the package to be analyzed, and constructing a defining function set based on the defining function in any candidate dependency package;

[0026] if the calling function set is a subset of the defining function set, determining that the dependency relationship exists between the package to be analyzed and the candidate dependency package.

[0027] In the implementation process, the comparison between the two sets can accurately determine whether the calling is established.

[0028] Further, in some embodiments, the method further comprises:

[0029] generating a dependency graph based on the dependency relationship, wherein a node of the dependency graph indicates a Python package, and an edge of the dependency graph indicates a dependency relationship between the Python packages.

[0030] In the implementation process, when a security problem occurs, all affected Python packages can be accurately found by using the dependency graph, thereby improving the efficiency of problem troubleshooting.

[0031] Further, in some embodiments, the method further comprises:

[0032] The generating of the dependency graph based on the dependency relationship comprises:

[0033] generating a node corresponding to the package to be analyzed and a node corresponding to each candidate dependency package;

[0034] if it is determined that the dependency relationship exists between the package to be analyzed and any candidate dependency package, generating an edge between the node corresponding to the package to be analyzed and the node corresponding to the candidate dependency package, wherein the edge points to the node corresponding to the candidate dependency package.

[0035] In the implementation process, a solution for generating a dependency graph is provided.

[0036] Further, in some embodiments, the method further comprises:

[0037] In a case where it is determined that the dependency relationship exists between the package to be analyzed and the candidate dependency package, reading an installation dependency parameter recorded in an installation guide file in the package to be analyzed;

[0038] In a case where the candidate dependency package is declared in the installation dependency parameter, determining that a first dependency relationship exists between the package to be analyzed and the candidate dependency package.

[0039] In a case where the candidate dependency package is not declared in the installation dependency parameter, determining that a second dependency relationship exists between the package to be analyzed and the candidate dependency package.

[0040] In the implementation process, the extracted dependency relationships are classified in a fine-grained manner to facilitate development and use.

[0041] Further, in some embodiments, the method further includes:

[0042] Indicating the first dependency relationship and the second dependency relationship based on different styles of edges of the dependency graph.

[0043] In the implementation process, the two dependency relationships are indicated by different styles, so that the dependency graph can clearly show the two dependency relationships, facilitating development and use of the user.

[0044] Further, in some embodiments, the style includes at least one of the following:

[0045] Line type, thickness, and color.

[0046] In the implementation process, the user can clearly distinguish different dependency relationships by different line types, thicknesses, and / or colors, thereby improving user experience.

[0047] In a second aspect, an apparatus for detecting a Python package dependency relationship is provided, and the apparatus includes:

[0048] An information acquisition module configured to acquire module information of all imported modules in a package to be analyzed;

[0049] A candidate determination module configured to determine a candidate dependency package corresponding to the package to be analyzed according to the module information;

[0050] A function acquisition module configured to acquire a calling function in the package to be analyzed and a defined function in the candidate dependency package; the defined function is a function defined using a keyword def.

[0051] A function comparison module configured to determine whether a dependency relationship exists between the package to be analyzed and the candidate dependency package according to a comparison result between the calling function and the defined function.

[0052] In a third aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor implements the steps of the method according to any one of the first aspect when running the computer program.

[0053] In a fourth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores instructions, and the instructions, when running on a computer, cause the computer to perform the method according to any one of the first aspect.

[0054] In a fifth aspect, a computer program product is provided, and the computer program product, when running on a computer, causes the computer to perform the method according to any one of the first aspect.

[0055] Other features and advantages of the present application will be described in the following description and other parts, or some features and advantages can be known or determined from the description without any doubt, or can be obtained by practicing the above-mentioned technology disclosed in the present application.

[0056] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the following preferred embodiments are specifically described, and the accompanying drawings are described in detail as follows. BRIEF DESCRIPTION OF DRAWINGS

[0057] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation to the scope, and for those skilled in the art, other related drawings can also be obtained without any creative labor on the basis of these drawings.

[0058] Figure 1 A flow chart of a method for detecting Python package dependency relationship provided by the embodiments of the present application;

[0059] Figure 2 A schematic diagram of a workflow of extracting dependency relationship between Python packages by a host provided by the embodiments of the present application;

[0060] Figure 3 A block diagram of a device for detecting Python package dependency relationship provided by the embodiments of the present application;

[0061] Figure 4 A structural block diagram of an electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0062] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.

[0063] It should be noted that similar reference numbers and letters represent similar items in the following drawings, and therefore, once an item is defined in one drawing, it need not be further defined and explained in subsequent drawings. Meanwhile, in the description of the present application, the terms "first", "second", and the like are only used to distinguish description, and cannot be understood as indicating or implying relative importance.

[0064] As described in the background, there is a problem in the related art that the incomplete Python package dependency graph cannot completely exclude the security risks caused by the attack on the dependent package. Based on this, the embodiments of the present application provide a method for detecting Python package dependency to solve this problem.

[0065] First, some professional terms involved in the embodiments of the present application are explained:

[0066] Module: a collection of a series of related codes organized together on the basis of functions and classes. In Python, a module is a source program file with a py extension, which contains Python object definitions and Python statements.

[0067] Package: a container for modules. A package can contain one or more modules. In Python, in order to facilitate calling, some modules with similar functions are usually organized together, or a more complex module is split into multiple components. These source program files can be placed in the same folder and managed according to Python rules. Such a folder and files in it are called a package.

[0068] PyPI (Python Package Index): an official software repository established to unify the release of Python third-party packages. Various software developed by developers all over the world based on Python is shared in a standard package form, which is called a third-party package. Users can download suitable third-party packages from PyPI for installation according to their needs.

[0069] Next, the embodiments of the present application are introduced:

[0070] As shown in Figure 1 , a method for detecting Python package dependency is provided, which comprises the following steps: Figure 1is a flowchart of a method for detecting Python package dependencies provided by an embodiment of the present application. The method can be applied to a terminal or a server. The terminal can be various electronic devices, including but not limited to smartphones, tablet computers, laptop portable computers, desktop computers, and the like. The server can be a single server or a distributed server cluster composed of multiple servers. The terminal or server provides a Python data analysis environment, which includes a software part and a hardware part. The software part mainly includes operating systems such as Windows, Linux, and the like, and the hardware part mainly includes computing resources, storage resources, and the like, such as CPUs (Central Processing Units), memories, hard disks, and the like. It should be noted that the terminal / server can also be implemented as multiple software or software modules, or can also be implemented as a single software or software module, and the present application does not limit this.

[0071] The method comprises:

[0072] In step 101, module information of all imported modules in the to-be-analyzed package is acquired.

[0073] The to-be-analyzed package mentioned in this step can refer to a Python third-party package. When developing with Python, a user can save his own development time by calling Python packages written by other developers. These Python packages can be downloaded by the user from PyPI to the local, or can be obtained by the user from other sources, such as the package author.

[0074] The imported module mentioned in this step can refer to a module imported using an import statement. In Python, the import statement is used to call a module. When a user needs to use a function or a class defined in a Python third-party package in the code, the corresponding module can be imported using the import statement in the code. The module information of the imported module can be related information describing the attributes and functions of the imported module, such as the module name of the imported module, executable statements contained therein, and the like.

[0075] In some embodiments, the module information can be obtained through an abstract syntax tree. The abstract syntax tree, which can also be referred to as a syntax tree, is an abstract representation of the syntax structure of source code. It represents the syntax structure of a programming language in a tree-like form, and each node on the tree represents a structure in the source code. After decompressing the Python package, the source code thereof can be accessed, and the source code can be converted into an abstract syntax tree through an AST module provided by Python, and then the required information can be obtained. Of course, in other embodiments, the abstract syntax tree of the to-be-analyzed package can also be converted from the source code of the to-be-analyzed package through other compilers, and the present application does not limit this.

[0076] It should be noted that if there is no imported module in the to-be-analyzed package, it indicates that the to-be-analyzed package is a completely independent package and does not depend on other Python packages.

[0077] In step 102, according to the module information, the candidate dependent package corresponding to the to-be-analyzed package is determined.

[0078] The candidate dependent package mentioned in this step refers to all possible dependent packages of the to-be-analyzed package. Since the module information of the imported module is related information describing the attributes and functions of the imported module, the third-party package corresponding to the imported module can be determined through the module information, and the third-party package may be a dependent package of the to-be-analyzed package.

[0079] In some embodiments, the module information can include the package name corresponding to the imported module; this step can include: searching for a third-party package corresponding to the package name according to the package name corresponding to the imported module; and determining the searched third-party package as the candidate dependent package corresponding to the to-be-analyzed package. Python third-party packages have unique package names, that is, the name of the package itself. The package name corresponding to the imported module refers to the package name of the third-party package corresponding to the imported module, and through the package name, the corresponding third-party package can be searched on PyPI to serve as the candidate dependent package corresponding to the to-be-analyzed package. For example, the to-be-analyzed package is package 1, which contains imported module A and imported module B, if the third-party package corresponding to the imported module A is package 2 and the third-party package corresponding to the imported module B is package 3, then through the package name corresponding to each of the two imported modules, package 2 and package 3 can be searched on PyPI, and then package 2 and package 3 are determined as the candidate dependent package corresponding to package 1.

[0080] It should be noted that at least one candidate dependent package can be determined according to the module information of one imported module. PyPI does not allow duplicate package names, but does not limit module names, so two different third-party packages can have the same module name. That is, the module name of one imported module can correspond to multiple package names, in which case, all package names corresponding to the module name of the imported module can be searched on PyPI to search for the corresponding third-party packages, and these searched third-party packages are all used as candidate dependent packages. The functions defined by modules with the same module name are not necessarily the same, so these candidate dependent packages can be screened through the function comparison in the subsequent steps to determine the packages that have a dependent relationship with the to-be-analyzed package.

[0081] In step 103, the calling function in the to-be-analyzed package and the defined function in the candidate dependent package are obtained; the defined function is a function defined using the keyword def.

[0082] In Python, a function refers to an organized, reusable piece of code that implements a single or associated function. The defining function mentioned in this step refers to a function defined using the keyword def (short for define), while the calling function refers to a function that has been defined and used in a Python package. It should be noted that the calling function can be a calling function related to the imported module, and correspondingly, the defining function can be a defining function related to the imported module; or in some scenarios, the calling function can also be all calling functions within the package to be analyzed, and correspondingly, the defining function can also be all defining functions within the candidate dependent package.

[0083] In some embodiments, the calling function mentioned in this step can be obtained through the abstract syntax tree of the package to be analyzed, and the defining function can be obtained through the abstract syntax tree of the candidate dependent package. That is, the definition and use of functions within a Python package can be obtained through the abstract syntax tree. Following the previous example, the package to be analyzed is package 1, and one of the candidate dependent packages of the package to be analyzed is package 2. The source code of package 1 can be converted into an abstract syntax tree through the AST module, and the calling functions within package 1 can be obtained by traversing the abstract syntax tree. At the same time, the source code of package 2 is also converted into an abstract syntax tree through the AST module, and the defining functions within package 2 are obtained by traversing the abstract syntax tree.

[0084] Specifically, the defining function can be obtained based on the following manner: searching for FunctionDef nodes in the abstract syntax tree of the candidate dependent package, each node of the abstract syntax tree representing a structure in the source code of the candidate dependent package; and determining the searched FunctionDef nodes as the defining functions within the candidate dependent package. The abstract syntax tree is an abstract representation of the syntax structure of the source code, and each node on the tree represents a structure in the source code. In the abstract syntax tree of the candidate dependent package, the function defined using the keyword def generates a FunctionDef node, and therefore, the defining functions within the candidate dependent package can be obtained by searching for FunctionDef nodes in the abstract syntax tree of the candidate dependent package.

[0085] The calling function can be obtained based on the following manner: searching for a Call node in each node in an abstract syntax tree of the package to be analyzed, each node in the abstract syntax tree representing a structure in source code of the package to be analyzed; and determining the function searched in the searched Call node as the calling function in the package to be analyzed. The Call node herein is generated by a Call function in the source code, and the essence of the Call function is to turn a class into a function, so that an instance of the class can be called like a function. Therefore, in the abstract syntax tree of the package to be analyzed, when a Call node appears in an arbitrary node and a function is searched in the Call node, the searched function is a calling function in the package to be analyzed.

[0086] In step 104, it is determined whether there is a dependency relationship between the package to be analyzed and the candidate dependent package according to a comparison result between the calling function and the defining function.

[0087] This step refers to that according to a comparison result between the calling function in the package to be analyzed and the defining function in the candidate dependent package, it can be determined whether the calling of the package to be analyzed on the candidate dependent package is established, if the calling is established, it can be determined that the package to be analyzed depends on the candidate dependent package, otherwise it can be determined that the package to be analyzed does not depend on the candidate dependent package.

[0088] Specifically, in some embodiments, this step can include: constructing a calling function set based on the calling function in the package to be analyzed, and constructing a defining function set based on the defining function in any candidate dependent package; if the calling function set is a subset of the defining function set, it is determined that there is a dependency relationship between the package to be analyzed and the candidate dependent package. That is, the calling function in the package to be analyzed and the defining function in the candidate dependent package are regarded as two sets, and then the two sets are compared, when it is found that all calling functions can be hit in the defining function, it is indicated that the calling is established, and then it can be determined that there is a dependency relationship between the package to be analyzed and the candidate dependent package, that is, the package to be analyzed depends on the candidate dependent package.

[0089] Corresponding to the "strong dependency" mentioned in the background, when a py file in a Python package references other modules, and the referenced modules do not declare the corresponding packages in the setup.py, the corresponding packages of the referenced modules will not be installed, and the user will report an error when using the code. Such dependency can be called "weak dependency". The related art does not consider such dependency when extracting the dependency relationship between Python packages. However, in actual application, when a package is attacked, not only the packages "strongly dependent" on the package will be affected, but also the packages "weakly dependent" on the package will be affected, and at this time, if only the former is processed, the latter still has the risk of development.

[0090] In the embodiment of the present application, all possible dependent packages are found through the module information of all imported modules in the to-be-analyzed package, and then it is determined whether there is a dependent relationship by comparing the calling functions in the to-be-analyzed package with the defined functions in each possible dependent package. In this way, the extracted dependent relationship covers both the "strong dependent" relationship of the dependent package declared in the plaintext and the "weak dependent" relationship of the dependent package not declared. Thus, the accurate and comprehensive extraction of the dependent relationship between Python packages is realized, and the user can completely eliminate the security risks caused by the dependent attacked package.

[0091] In addition, in some embodiments, the method can further include generating a dependency graph based on the dependent relationship, wherein the nodes of the dependency graph indicate the Python packages, and the edges of the dependency graph indicate the dependent relationship between the Python packages. That is, after the dependent relationship between the Python packages is extracted, a dependency graph can be generated based on each dependent relationship. Specifically, the generation process of the dependency graph can include generating a node corresponding to the to-be-analyzed package and a node corresponding to each candidate dependent package, and if it is determined that there is a dependent relationship between the to-be-analyzed package and any candidate dependent package, generating an edge between the node corresponding to the to-be-analyzed package and the node corresponding to the candidate dependent package, wherein the edge points to the node corresponding to the candidate dependent package.

[0092] Since the dependency graph generated by the embodiment takes into account both "strong dependent" and "weak dependent", when a security problem occurs, all affected Python packages can be accurately found using the dependency graph, thereby improving the efficiency of problem troubleshooting.

[0093] In addition, after the dependency relationship is extracted, the dependency relationship can be further divided in a fine-grained manner to facilitate development and use. Therefore, in some embodiments, the method further comprises: when it is determined that there is a dependency relationship between the package to be analyzed and the candidate dependency package, reading an installation dependency parameter recorded in an installation guide file in the package to be analyzed; if the candidate dependency package is declared in the installation dependency parameter, determining that there is a first dependency relationship between the package to be analyzed and the candidate dependency package; if the candidate dependency package is not declared in the installation dependency parameter, determining that there is a second dependency relationship between the package to be analyzed and the candidate dependency package. In Python, the installation guide file refers to a setup.py file, and the installation dependency parameter refers to an install_requires parameter in the setup.py file. When the candidate dependency package is declared in the installation dependency parameter, the candidate dependency package will be installed synchronously when the package to be analyzed is installed, and at this time, the package to be analyzed strongly depends on the candidate dependency package, that is, there is a first dependency relationship; when the candidate dependency package is not declared in the installation dependency parameter, that is, the package to be analyzed does not declare the candidate dependency package in the setup.py file, the candidate dependency package will not be installed synchronously when the package to be analyzed is installed, and at this time, the package to be analyzed weakly depends on the candidate dependency package, that is, there is a second dependency relationship.

[0094] Further, after the extracted dependency relationship is subdivided into a first dependency relationship and a second dependency relationship, the first dependency relationship and the second dependency relationship can be indicated based on different styles of edges of the dependency graph. The style here can include at least one of line type, thickness, color, etc. For example, when it is determined by the scheme of the present embodiment that package 1 “strongly depends” on package 2 and package 1 “weakly depends” on package 3, in the generated dependency graph, the edge between the two nodes of package 1 and package 2 points to package 2, and the line type of the edge is a solid line, indicating that package 1 “strongly depends” on package 2, while the edge between the two nodes of package 1 and package 3 points to package 3, and the line type of the edge is a dashed line, indicating that package 1 “weakly depends” on package 3. In this way, the dependency graph can clearly show the two dependency relationships, facilitating the development and use of users.

[0095] In order to make a more detailed description of the scheme of the present application, a specific embodiment is introduced as follows:

[0096] In the present embodiment, a user develops a Python project on a host, which involves multiple Python packages. Since the user will install any possible package when receiving an error of missing dependency, which will bring risks to development, it is necessary to build a dependency graph based on the dependency relationship between Python packages. After applying the scheme of the present application, the host can automatically extract the dependency relationship between Python packages.

[0097] Taking the PyPI package currently being analyzed as an example, the package is a Python package with a package name of test-pack-0.1 and a packaging format of tar.gz, the workflow of the host for extracting the inter-Python package dependency relationship is as shown in Figure 2

[0098] S201, discovering, through an abstract syntax tree, that a module requests (labeled as 22 in the figure) is called in the package test-pack-0.1 (labeled as 21 in the figure);

[0099] S202, determining, according to the module information of the module requests, that the third-party package corresponding to the module requests is the package requests-2.28.0 (a Python package with a package name of requests-2.28.0 and a packaging format of tar.gz, labeled as 23 in the figure);

[0100] S203, acquiring, through an abstract syntax tree, function calls related to the module requests, to thereby construct a calling function set 24, the calling function set 24 containing two calling functions of requests.get and requests.post;

[0101] S204, acquiring, through an abstract syntax tree, function definitions in the package requests-2.28.0, to thereby construct a defining function set 25, the defining function set 25 containing functions of requests.get, requests.post, requests.head, and the like;

[0102] S205, comparing the calling function set 24 and the defining function set 25;

[0103] S206, detecting that all calling functions in the calling function set 24 can be hit in the defining functions contained in the defining function set 25, determining that the calling is established, and determining that the package test-pack-0.1 depends on the package requests-2.28.0.

[0104] ​Through the above process, the host can more accurately and comprehensively extract the dependency relationship between Python packages. Through the dependency graph constructed by using the dependency relationship, when a security problem occurs, all affected Python packages can be accurately pointed out. For example, when the boundary function requests.get in the package requests-2.28.0 is illegally tampered with and causes information to be transmitted outside, if the package test-pack-0.1 uses the function, there will be a security risk. Based on the extracted dependency relationship, the user can discover that the package test-pack-0.1 is also an affected package in a timely manner, and then appropriate processing measures can be taken. In this way, the security risk caused by the dependency of the attacked package is effectively ruled out, and the security of development is improved.

[0105] Corresponding to the embodiments of the foregoing method, the application further provides an embodiment of a device for detecting Python package dependency relationship and an application terminal thereof:

[0106] As shown in Figure 3 , Figure 3 is a block diagram of a device for detecting Python package dependency relationship provided by an embodiment of the application. The device comprises:

[0107] An information acquisition module 31 is configured to acquire module information of all imported modules in a to-be-analyzed package.

[0108] A candidate determination module 32 is configured to determine a candidate dependent package corresponding to the to-be-analyzed package according to the module information.

[0109] A function acquisition module 33 is configured to acquire a calling function in the to-be-analyzed package and a boundary function in the candidate dependent package; the boundary function is a function defined using a keyword def.

[0110] A function comparison module 34 is configured to determine whether there is a dependency relationship between the to-be-analyzed package and the candidate dependent package according to a comparison result between the calling function and the boundary function.

[0111] The implementation process of the functions and roles of each module in the above device is specifically described in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0112] The application further provides an electronic device, please see Figure 4 , Figure 4A structural block diagram of an electronic device is provided in the embodiments of the present application. The electronic device can include a processor 410, a communication interface 420, a memory 430, and at least one communication bus 440. The communication bus 440 is used to realize direct connection communication among the components. The communication interface 420 of the electronic device in the embodiments of the present application is used to communicate signaling or data with other node devices. The processor 410 can be an integrated circuit chip with signal processing capability.

[0113] The processor 410 described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; or can be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a ready programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The disclosed methods, steps and logic block diagrams in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor 410 can also be any conventional processor.

[0114] The memory 430 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read only memory (PROM), an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM), etc. The memory 430 stores computer readable instructions, which, when executed by the processor 410, enable the electronic device to perform the above-mentioned Figure 1 The method embodiments involve various steps.

[0115] Optionally, the electronic device can further include a storage controller, an input / output unit.

[0116] The memory 430, the storage controller, the processor 410, the peripheral interface, and the input / output unit are directly or indirectly electrically connected to each other to realize data transmission or interaction. For example, these elements can be electrically connected to each other through one or more communication buses 440. The processor 410 is used to execute executable modules stored in the memory 430, such as software function modules or computer programs included in the electronic device.

[0117] The input and output unit is used to provide a user with a creation task and create a selectable period or preset execution time for starting the task to realize the interaction between the user and the server. The input and output unit can be, but is not limited to, a mouse, a keyboard and the like.

[0118] It can be understood that Figure 4 The structure shown is only schematic, and the electronic device can further include more or fewer components than those shown, or have a different configuration of components than those shown. Figure 4 The components shown in the figures can be implemented in hardware, software or a combination thereof. Figure 4 The components shown in the figures can be implemented in hardware, software or a combination thereof. Figure 4 The components shown in the figures can be implemented in hardware, software or a combination thereof.

[0119] The embodiments of the present application also provide a storage medium, which has instructions stored thereon, when the instructions are run on a computer, the computer program is executed by a processor to implement the method of the method embodiments. To avoid repetition, it will not be described here.

[0120] The present application also provides a computer program product, which, when run on a computer, causes the computer to execute the method of the method embodiments.

[0121] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are only schematic, for example, the flowcharts and block diagrams in the drawings show the possible implementation architectures, functions and operations of the devices, methods and computer program products according to the embodiments of the present application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment or a part of code, which contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different orders from those shown in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for executing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0122] In addition, each functional module in the various embodiments of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0123] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts of the prior art that make contributions or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0124] The above merely provides an example of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application. It should be noted that similar reference numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0125] The above merely provides an example of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application. It should be noted that similar reference numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0126] It should be noted that, in this document, the terms such as first and second are used merely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the processes, methods, articles or devices including a series of elements not only include those elements, but also include other elements not explicitly listed or inherent to such processes, methods, articles or devices. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

Claims

1. A method for detecting Python package dependencies, characterized in that, include: Obtain module information for all imported modules within the package to be analyzed; Based on the module information, determine the candidate dependency packages corresponding to the package to be analyzed; Obtain the calling functions within the package to be analyzed, and the delimiting functions within the candidate dependency packages; The defined function is a function defined using the keyword `def`. The called function is a predefined function used in the Python package; Based on the comparison results between the calling function and the delimiting function, it is determined whether there is a dependency relationship between the package to be analyzed and the candidate dependency package; The step of determining whether a dependency relationship exists between the package to be analyzed and the candidate dependency package based on the comparison result between the calling function and the delimiting function includes: A set of calling functions is constructed based on the calling functions within the package to be analyzed, and a set of delimiting functions is constructed based on the delimiting functions within any candidate dependency package; If the set of called functions is a subset of the defined set of functions, it is determined that there is a dependency relationship between the package to be analyzed and the candidate dependency package.

2. The method according to claim 1, characterized in that, The module information includes the package name corresponding to the imported module; determining the candidate dependency package corresponding to the package to be analyzed based on the module information includes: Based on the package name corresponding to the import module, search for the third-party package corresponding to the package name; The third-party packages found in the search are identified as candidate dependency packages corresponding to the package to be analyzed.

3. The method according to claim 1, characterized in that, The calling function is obtained through the abstract syntax tree of the package to be analyzed; the delimiting function is obtained through the abstract syntax tree of the candidate dependency packages.

4. The method according to claim 3, characterized in that, The boundary function is obtained based on the following method: Search the abstract syntax tree of the candidate dependency package for FunctionDef nodes, where each node of the abstract syntax tree represents a structure in the source code of the candidate dependency package; The FunctionDef node found in the search is identified as the bounding function within the candidate dependency package.

5. The method according to claim 3, characterized in that, The calling function is obtained in the following way: Search for Call child nodes in each node of the abstract syntax tree of the package to be analyzed, where each node of the abstract syntax tree represents a structure in the source code of the package to be analyzed; The functions retrieved from the searched Call sub-nodes are identified as the calling functions within the package to be analyzed.

6. The method according to claim 1, characterized in that, Also includes: A dependency graph is generated based on the dependency relationships, where the nodes of the dependency graph indicate Python packages and the edges of the dependency graph indicate the dependencies between the Python packages.

7. The method according to claim 6, characterized in that, The generation of the dependency graph based on the dependency relationship includes: Generate the node corresponding to the package to be analyzed and the node corresponding to each candidate dependency package; If it is determined that there is a dependency relationship between the package to be analyzed and any candidate dependency package, an edge is generated between the node corresponding to the package to be analyzed and the node corresponding to the candidate dependency package, and the edge points to the node corresponding to the candidate dependency package.

8. The method according to claim 6, characterized in that, Also includes: When it is determined that there is a dependency relationship between the package to be analyzed and the candidate dependency package, the installation dependency parameters recorded in the installation guidance file in the package to be analyzed are read; If the installation dependency parameters include a declaration of the candidate dependency package, it is determined that there is a first dependency relationship between the package to be analyzed and the candidate dependency package. If the candidate dependency package is not declared in the installation dependency parameters, it is determined that there is a second dependency relationship between the package to be analyzed and the candidate dependency package.

9. The method according to claim 8, characterized in that, Also includes: The first dependency relationship and the second dependency relationship are indicated based on the different edge styles of the dependency graph.

10. The method according to claim 9, characterized in that, The style includes at least one of the following: Line type, thickness, and color.

11. An apparatus for detecting Python package dependencies, characterized in that, include: The information acquisition module is used to acquire module information of all imported modules in the package to be analyzed; The candidate determination module is used to determine the candidate dependency packages corresponding to the package to be analyzed based on the module information. The function acquisition module is used to acquire the calling functions within the package to be analyzed, and the delimited functions within the candidate dependency packages; the delimited functions are functions defined using the keyword `def`; the calling functions are predefined functions used in Python packages. The function comparison module is used to determine whether there is a dependency relationship between the package to be analyzed and the candidate dependency package based on the comparison result between the calling function and the defining function; The function comparison module is specifically used for: A set of calling functions is constructed based on the calling functions within the package to be analyzed, and a set of delimiting functions is constructed based on the delimiting functions within any candidate dependency package; If the set of called functions is a subset of the defined set of functions, it is determined that there is a dependency relationship between the package to be analyzed and the candidate dependency package.

12. A computer-readable storage medium, characterized in that, It stores a computer program thereon, which, when executed by a processor, implements the method as described in any one of claims 1 to 10.

13. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method as described in any one of claims 1 to 10.

Images