A verification-based Python function argument type inference method

Abstract

This invention discloses a verification-based method for inferring Python function parameter types. The method involves parsing the input Python source file to extract function and parameter information; constructing an initial set of candidate parameter types; identifying the behavioral characteristics of function parameters within the function body based on static analysis and filtering the candidate type set accordingly; adding type annotations to the function parameters and introducing a static type checker to verify the candidate types; and integrating the verified types to obtain the final parameter type inference result. By introducing a Python static type checker into the type inference process, this invention effectively improves the accuracy and comprehensiveness of function parameter type inference results. It can reliably infer Python function parameter types even in the absence of call point context or explicit type annotations, overcoming the inaccurate and incomplete inference results of existing methods, and has high engineering practical value.

Description

Technical Field

[0001] This invention belongs to the field of software analysis technology, specifically relating to a verification-based method for inferring Python function parameter types. It is applicable to automatically inferring the types of function parameters in target Python source files and generating structured inference results. This method combines static analysis to extract constraints with type checker verification to automatically infer the types of function parameters in target Python source files. The method can be invoked via command line to generate structured type inference results for subsequent analysis, annotation, or application. Background Technology

[0002] With the continuous expansion of software system scale and the popularization of rapid iterative development models, the correctness, maintainability, and readability of program code have gradually become important issues in the field of software engineering. Python, as a typical dynamically typed programming language, has been widely used in data analysis, artificial intelligence, automation scripts, and backend services due to its advantages such as concise syntax, high development efficiency, and low learning cost. Python employs a mechanism where "variables do not need to be explicitly declared in terms of type; the type is determined by the runtime value," enabling developers to quickly complete program writing even without complete type information, thus significantly improving development flexibility and prototyping efficiency.

[0003] However, due to Python's dynamic typing, type information is incomplete during the code writing phase. The compiler or interpreter cannot statically verify the type validity of variables and function parameters; type checking is typically delayed until runtime. If a type mismatch error occurs during runtime, it often leads to program exceptions or even system failures, resulting in high costs for locating and debugging related issues. Especially in scenarios with large codebases or collaborative development, ambiguous type contracts can easily lead to potential logical errors, reducing code maintainability and reliability.

[0004] To alleviate these problems, the industry has gradually introduced techniques such as type annotation and type inference to compensate for the shortcomings of dynamically typed languages ​​in terms of type safety. Function parameter types, as a crucial component describing function interfaces, play a key role in understanding program behavior, obtaining accurate API information, and performing program correctness analysis. However, in actual development, many Python function parameters lack explicit type declarations or rely solely on non-standard and incomplete comments, resulting in a general lack of type information for function parameters. Furthermore, since function parameters often play multiple roles within the function body, their type constraints are implicit in the complex control and data flow, making type inference for function parameters more difficult than for return values ​​or local variables.

[0005] While some existing Python type inference tools can provide a certain level of inference results in specific scenarios, they often struggle to balance accuracy and coverage when lacking sufficient context information or type annotation information. This can lead to incomplete or incorrect type inferences, failing to meet the needs of practical development and engineering applications. Therefore, there is an urgent need for a technical solution that can comprehensively infer Python function parameter types while ensuring correctness. This would provide developers with reliable parameter type reference information, reducing the workload of manual type annotation, improving the accuracy of type annotation, and ultimately enhancing code readability, maintainability, and overall software quality. Summary of the Invention

[0006] Purpose of the invention: This invention addresses the shortcomings of existing Python type inference tools in the function parameter type inference process by providing a verification-based Python function parameter type inference method. This method solves the problems of insufficient accuracy and limited coverage of inference results caused by the lack of explicit type annotations for function parameters and the fact that type constraints are implicit in the complex program context. It also meets the requirements of practical engineering applications for the correctness and completeness of parameter type information.

[0007] Technical solution: A verification-based method for inferring Python function parameter types, comprising the following steps: Step 1: Input the target Python source file as the object to be analyzed; Step 2: Parse the target Python source file and extract the set of functions defined in the file. and the corresponding set of function parameter lists ; Step 3: Collect candidate types of function parameters and generate an initial set of candidate types. ; Step 4: Traverse the function Abstract syntax tree, analyze parameters The behavioral characteristics exhibited within the function body form a set of parameter behavior constraints. and parameters Initial candidate type set Filtering is performed to obtain a set of filtered candidate types. ; Step 5: Use the filtered candidate types for functions Type annotations are added to the parameters, and a static type checker is used for validation to filter out the set of parameter types that pass the validation. ; Step Six: For the set of verified types Integrate and reduce functions to generate functions. Parameter type inference results .

[0008] The specific measures taken include: In step two, the present invention performs syntax parsing on the target Python source file, extracts all functions defined therein, and forms a function set. ;for Extract its formal parameter set During the extraction process, this invention records the following information for functions and parameters respectively: For functions, the function name is recorded; when the function is a member method of a class, the function name is represented in the form of "class name.function name"; the project name to which the function belongs and the name of the source file where it is located are also recorded; for parameters, the parameter name, the name of the function to which it belongs, and the line number in the source file where it is located are recorded; when a parameter has a default value, the literal or expression corresponding to the default value is further recorded for subsequent type inference reference. The function set... and its parameter list set As the basic input for subsequent type inference processes.

[0009] In step three, the present invention collects and constructs an initial candidate type set from multiple sources. Using existing tools like AexPy and based on Python's official open-source type annotation repository, typeshed, this invention filters and extracts available built-in types from the Python standard library to obtain a set. ; Extract classes from the current project and its dependency environment based on import analysis to construct a user-defined type set. Take the union of the two sets as the initial candidate type set. For step two and its parameter set The initial set of candidate parameter types is uniformly set to This serves as the initial type field to be selected in subsequent steps.

[0010] In step four, the present invention iterates through the function. The abstract syntax tree is used to extract the behavioral constraints of the parameters and filter candidate types accordingly. Define its set of behavioral constraints as Define behavior filtering operators. Its function is to retain types from the type set that satisfy a given behavioral constraint, i.e. The result after filtering This will be used as the input for step five.

[0011] In step five, the present invention applies the filtered candidate type set Based on this, a verification-based type filtering mechanism is introduced. For and its parameter set The following verification strategy is adopted: If (Single-parameter function), let ,for Construct with type annotation The function test case, and calls the static type checker PyRight for validation (defining the validation operation). ), to obtain the set of validated parameter types ;like (For multi-parameter functions), first construct a combination of candidate parameter types. and to (in Indicates the first Construct function test cases with corresponding type annotations (considering candidate types for each parameter), call PyRight for verification, and obtain a set of verified type combinations. To improve efficiency and pruning capabilities during the verification process, this invention parallelizes the verification operations (multi-core parallel execution) and constructs a class inheritance graph based on type inheritance relationships. The verification results will be in Propagation and pruning are performed on the parameters to be verified, thereby dynamically updating the set of parameters to be verified and reducing the verification space during the iteration.

[0012] In step six, the present invention verifies the set of parameter types. Integration and simplification processes were performed. Specifically, for... Application of reduction operators Merging rules based on type inheritance relationships, when When both an (abstract) base class type (composition) and its corresponding subclass type (composition) exist, the base class type (composition) is retained to reduce redundancy in the result, thus improving the overall performance. Generate the final parameter type inference result. The results are ultimately written to a specified output path in JSON file format, which can be used for type annotation generation, program analysis, or downstream engineering applications.

[0013] The beneficial effects of this invention are: It can provide highly accurate and comprehensive Python function parameter type inference results. By systematically analyzing the behavior of function parameters in the program and combining it with type checker verification, it effectively reduces erroneous type inference and type omissions, providing developers with reliable type reference information.

[0014] Even in the absence of explicit type annotations or calling context information, it is still possible to effectively infer the type of function parameters, improve the type analyzability of dynamically typed languages ​​in complex engineering scenarios, and enhance their applicability to real-world project code.

[0015] By automating the parameter type inference process, the workload of manual type labeling and the risk of human error are significantly reduced, the consistency and accuracy of type labeling are improved, and the subsequent maintenance costs are reduced.

[0016] The inference results are comprehensive and interpretable, providing support for understanding function interfaces, generating API documentation, and analyzing program correctness, thereby improving code readability and maintainability.

[0017] This invention provides a command-line tool interface that facilitates integration with existing development processes, automated analysis or evaluation systems, supports batch analysis and engineering applications, and has high practical value and promotional significance. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating how the target Python file function parameter types are inferred in this invention. Figure 2 This is a flowchart of step four in this invention; Figure 3 This is a flowchart of step five in this invention; Figure 4 The following are example Python source files and generated result files for using this invention to perform parameter type inference. Detailed Implementation

[0019] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading the present invention, any modifications of the present invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0020] Appendix Figure 1 The flowchart illustrating the present invention's function parameter type inference for a target Python file mainly includes the following steps: Step 1: Input the target Python source file as the object to be analyzed; Step 2: Parse the target Python source file and extract the set of functions defined in the file. and the corresponding set of function parameter lists ; This invention first performs abstract syntax tree parsing on the target Python source file, analyzing and processing the source file using function definitions as the basic unit. Specifically, it extracts all functions defined in the source file to form a function set. ;for Extract its formal parameter set For each function This invention records the function name, the project name, and the file name; when the function is a member method, the function name is represented as "class name.function name" to clearly specify the context information of its class; for each parameter This invention records the parameter name, the name of the function to which it belongs, and the line number. When a parameter has a default value, this invention records the literal or expression corresponding to the default value as auxiliary reference information in the parameter type inference process. Through the above steps, this invention constructs an analysis object containing function parameter identifiers and location information, providing necessary input conditions for subsequent static analysis based on parameter behavior characteristics and verification processes based on type checkers.

[0021] Step 3: Collect candidate types of function parameters and generate an initial set of candidate types. ; This invention constructs an initial set of candidate types for function parameters through multi-source analysis. Includes a collection of built-in types and user-defined type collection For the collection of built-in types, this invention utilizes the relevant APIs provided by the existing tool AexPy and combines them with the type annotation repository typeshed to extract public modules and types from the Python standard library, excluding private types and types in modules marked as deprecated. Furthermore, abstract or metatypes that lack specific semantic constraints and are difficult to use for function parameter type inference are also removed, thereby constructing a standardized, stable set of built-in types consistent with type annotation practices. For the collection of user-defined types, this invention extracts user-defined types by importing and analyzing the target Python source files and their dependencies. Specifically, it first collects all classes directly defined in the target source files and adds them to the user-defined type set. Subsequently, the local import statements in the source file are analyzed, all explicitly imported types are parsed and added. For global import statements, the source code of the imported package or module is further located, and all class definitions contained therein are extracted and uniformly incorporated. Through the above methods, this invention comprehensively constructs an initial set of candidate function parameter types that has a wide coverage and clear sources. This provides a foundation for subsequent candidate type filtering and confirmation based on static analysis and type verification.

[0022] Step 4: Traverse the function Abstract syntax tree, analyze parameters The behavioral characteristics exhibited within the function body form a set of parameter behavior constraints. and parameters Initial candidate type set Filtering is performed to obtain a set of filtered candidate types. ; Due to the duck typing feature of Python's dynamic typing system, the behavior of an object can reflect its type. Therefore, this invention is based on the objective function... Static analysis is performed on the Abstract Syntax Tree (AST) to identify function parameters. The behavioral characteristics exhibited within the function body, and the initial set of candidate types obtained in step three constrained by these behavioral characteristics. Filtering is performed. Specifically, this invention analyzes the function parameters during the traversal of the function AST. Relevant syntax nodes are used to identify and extract parameters. Type constraints on behavioral characteristics. These behavioral constraints constitute a set. These behavioral characteristics are highly representative and distinctive, enabling effective filtering and narrowing down the range of candidate types. For example, when an Attribute type node (such as "af") is accessed in the AST and its operation object is a parameter, [the following is a separate, unrelated sentence:] At that time, the judgment There are member access behaviors, and the names of the members accessed are recorded; when the Call node is accessed and When acting as the invoked object, determine It has callable behavior; when When participating in arithmetic operations, comparison operations, or iterative structures, determine the appropriate criteria. It possesses computable or iterable behavior. Subsequently, the present invention is based on the extracted... Behavioral characteristics of the initial set Filter the data to remove types that do not possess the corresponding behavioral capabilities, resulting in a filtered set of candidate types. This significantly narrows down the range of types to be verified, improving the efficiency of subsequent verification stages. (See attached image) Figure 2 A flowchart illustrating the above process is provided.

[0023] Step 5: Use the filtered candidate types for functions Type annotations are added to the parameters, and a static type checker is used for validation to filter out the set of parameter types that pass the validation. ; After completing static filtering based on behavioral features, this invention further... The remaining candidate types are verified to improve the accuracy of the final inference result. Specifically, this invention adds type annotations to function parameters and uses a static type checker to verify functions with type annotations, thereby filtering out types that fail the verification. (See attached...) Figure 3 A flowchart illustrating the steps described is provided.

[0024] To further reduce the number of candidate types in actual verification and improve overall efficiency, this invention introduces a class inheritance graph. The verification order is arranged according to the relationships between classes, and pruning is performed based on the verification results. The verification process is as follows: each round starts from... Several types were selected and verified in parallel on a multi-core processor. After obtaining the verification results, the results were then... It is disseminated online and updated accordingly. Then select a type from them for the next round of verification, and repeat the above process until all types have been verified.

[0025] Regarding the selection of the verification order, this invention is based on the candidate type. exist The pruning algorithm assigns a weighted score to types based on their position within the inheritance structure, considering both the number of base classes and subclasses. The verification order is then determined according to these scores, prioritizing types at key positions in the inheritance hierarchy to maximize pruning effectiveness. The verification results are then... The following rules apply during uppropagation: when a candidate type Upon successful verification, for All were determined to satisfy the type constraints and did not require repeated validation; otherwise, for All were deemed not to meet type constraints and were not included in the subsequent verification process. Furthermore, this invention constructs inter-class behavioral subset relationships based on type inheritance and Method Resolution Order (MRO) information. If type A passes verification, all types with a superset of behavior A pass; otherwise, all types with a subset of behavior A fail, further assisting in the propagation and pruning of verification results.

[0026] In selecting the verification tool, this invention comprehensively compares the verification efficiency and error detection capabilities of various mainstream Python static type checkers, and ultimately selects PyRight as the verification tool to ensure the accuracy of the verification process and the overall execution efficiency.

[0027] Step Six: For the set of verified types Integrate and reduce functions to generate functions. Parameter type inference results .

[0028] This step checks the set of types that have passed the static checker verification in the previous step. Application of reduction operators Get the type inference result set The rules are as follows: For functions ,like (Single-parameter function), when When a base class and its subclasses coexist, only the semantically broader and more abstract base class is retained; similarly, if... (Multi-parameter function), at this time For a set of validated parameter type combinations, when When there are both combinations composed of base classes and corresponding combinations composed of their subclasses, only the combination composed of base classes is retained.

[0029] Ultimately, this invention will be applicable to... Result set Write the result to the specified output path in JSON format as the parameter type inference result file. The result record contains function identification information (function name, project name, file name), parameter names, parameter position information (line number), and the corresponding structured data. (A set of single-parameter types or a set of combinations of multiple-parameter types) for downstream use. See attached... Figure 4 The paper demonstrates the JSON format result file and corresponding Python source file generated by parameter type inference using this invention. It includes examples of parameter type inference results for the single-parameter function must_computable and the multi-parameter function must_callable. It can be seen that the parameter type inference result for a single-parameter function (such as parameter a in the function must_computable) represents a list of compatible types for that parameter. For multi-parameter functions, since there may be type dependencies between parameters, the parameter type inference result (such as parameters a and b in the function must_callable) represents a list of compatible type combinations for the parameters.

Claims

1. A verification-based method for inferring the type of Python function parameters, characterized in that, Includes the following steps: Step 1: Input the target Python source file as the object to be analyzed; Step 2: Parse the target Python source file and extract the set of functions defined in the file. and the corresponding set of function parameter lists ; Step 3: Collect candidate types of function parameters and generate an initial set of candidate types. ; Step 4: Traverse the function Abstract syntax tree, analyze parameters The behavioral characteristics exhibited within the function body form a set of parameter behavior constraints. and parameters Initial candidate type set Filtering is performed to obtain a set of filtered candidate types. ; Step 5: Use the filtered candidate types for functions Type annotations are added to the parameters, and a static type checker is used for validation to filter out the set of parameter types that pass the validation. ; Step Six: For the set of verified types Integrate and reduce functions to generate functions. Parameter type inference results .

2. The verification-based Python function parameter type inference method according to claim 1, characterized in that, In step two, the target Python source file is parsed to extract all defined functions, forming a function set. ;for Extract its formal parameter set During the extraction process, the following information is recorded for functions and parameters respectively: For functions, record their function name; when the function is a member method of a class, the function name is represented in the form of "class name.function name"; also record the name of the project to which the function belongs and the name of the source file where it is located; For parameters, record their parameter name, the name of the function to which they belong, and the line number in the source file where they are located; when a parameter has a default value, further record the literal or expression corresponding to the default value for reference in subsequent type inference; the function set and its parameter list set As the basic input for subsequent type inference processes.

3. The verification-based Python function parameter type inference method according to claim 1, characterized in that, In step three, a collection is obtained by filtering and extracting available built-in types from the Python standard library. ; Extract classes from the current project and its dependency environment based on import analysis to construct a user-defined type set. Initial candidate type set ;for and its parameter set The initial set of candidate parameter types is uniformly set to .

4. The verification-based Python function parameter type inference method according to claim 1, characterized in that, In step four, by traversing the function The abstract syntax tree extracts the behavioral constraints of parameters and filters candidate types accordingly. Define its set of behavioral constraints as Define behavior filtering operators Its function is to retain types from the type set that satisfy a given behavioral constraint, i.e. .

5. The verification-based Python function parameter type inference method according to claim 1, characterized in that, In step five, in the filtered candidate type set Based on this, a verification-based type filtering mechanism is introduced; for and its parameter set The following verification strategy is adopted: If ,set up ,for Construct with type annotation The function test case is then used, and the static type checker PyRight is called to perform validation, resulting in a set of validated parameter types. ;like First, construct the combination of candidate parameter types. and to Construct function test cases with corresponding type annotations, call PyRight for validation, and obtain a set of valid type combinations. .

6. The verification-based Python function parameter type inference method according to claim 5, characterized in that, The verification process involves constructing a class inheritance graph based on the inheritance relationships between candidate types. The results of the type checker's verification are propagated in the class inheritance graph to reduce the number of types to be verified.

7. The verification-based Python function parameter type inference method according to claim 6, characterized in that, Validation results are propagated when a candidate type Upon successful verification, for All were determined to satisfy the type constraints and did not require repeated validation; otherwise, for All of them were judged to not meet the type constraints and were no longer included in the subsequent verification process.

8. The verification-based Python function parameter type inference method according to claim 1, characterized in that, In step six, the set of parameter types that have passed verification is... Integration and simplification processes were performed; Application of reduction operators Merging rules based on type inheritance relationships, when When both a base class type and its corresponding subclass type exist, the base class type is retained.

9. The verification-based Python function parameter type inference method according to claim 1, characterized in that, When function When it is a single-parameter function, the parameter type inference result Represented as a set of types corresponding to the parameters; when the function When it is a multi-parameter function, the parameter type inference result Represented as a set of validated parameter type combinations; Finally, the output is in JSON format as a file, which can be used for type annotation generation, program analysis, or subsequent engineering applications.

Images