A method, apparatus and storage medium for black-box code analysis and test case generation

Abstract

This invention relates to a method, apparatus, and storage medium for black-box code analysis and test case generation, applied in the field of software testing technology. It includes: non-intrusive acquisition of variable change trajectories and execution paths within black-box code, making previously invisible internal logic states transparent, enabling testers to accurately locate intermediate logic errors, and solving the problem of traditional black-box testing's inability to observe internal states. Secondly, by systematically reconstructing the logical structure and execution path of the code, this application can accurately identify uncovered logical branches and intelligently reverse-engineer and generate supplementary test cases, overcoming the shortcomings of traditional black-box testing, such as reliance on experience and incomplete coverage, significantly improving the depth and efficiency of testing. Thirdly, this application uses a system tracing interface for monitoring, without modifying the target code's bytecode, avoiding the potential impact of intrusive methods on program performance and stability.

Description

Technical Field

[0001] This invention relates to the field of software testing technology, specifically to a black-box code analysis and test case generation method, apparatus, and storage medium. Background Technology

[0002] In software development and maintenance, black-box testing is an important testing method that verifies the correctness of a program's functionality solely through its inputs and outputs. However, for compiled programs without source code, such as compiled Python files, traditional black-box testing methods have significant limitations. Testers cannot observe the program's internal state during runtime, such as changes in variable values, function call flows, and the execution of conditional branches. This makes it difficult to pinpoint the specific intermediate logic when an error occurs; furthermore, test case design heavily relies on the tester's experience, making it difficult to systematically guarantee effective coverage of the program's complex internal logic and boundary conditions. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a method, apparatus and storage medium for black-box code analysis and test case generation, which aims to solve the technical problem that the existing technology cannot effectively obtain the internal variable change trajectory and runtime logic structure of compiled black-box code without intruding on the target code and without significantly affecting its performance, thus leading to blind test case design, difficulty in locating intermediate logic errors and inability to guarantee test coverage.

[0004] According to a first aspect of the present invention, a method for black-box code analysis and test case generation is provided, the method comprising:

[0005] A tracing function is registered through the preset sys.settrace interface. The execution process of the black-box code is monitored non-intrusively through the tracing function. The tracing function is called when a preset event is triggered to obtain variable information of the relevant preset event.

[0006] Based on the changes in the variable values ​​of different variable names with the execution sequence and their corresponding bytecode offsets in the variable information of the relevant preset events, a variable trajectory mapping relationship is established;

[0007] By using the instruction mapping function of the preset decompilation tool, the variable trajectory mapping relationship is associated with the pre-acquired black box pseudocode line, so as to realize the understandable display of the variable trajectory;

[0008] Obtain the variable boundary values ​​based on the distribution of variable values ​​in the variable trajectory and the branch paths in the pre-acquired black-box pseudocode lines;

[0009] For branch paths that have not yet been covered, the combination of input variables that triggers the execution of that path is deduced in reverse based on the variable boundary values ​​and variable trajectories, and supplementary test cases are generated based on the combination of input variables.

[0010] Preferably,

[0011] The tracking function is invoked when a preset event is triggered, and the variable information of the relevant preset event is obtained, including:

[0012] When a preset event is triggered, the tracing function extracts runtime context data from the current execution stack frame. The context data includes local variables, global variables, and variable type information.

[0013] Preferably,

[0014] The preset events include at least: function call events, code line execution events, and function return events.

[0015] Preferably, it further includes:

[0016] By setting a decorator at the entry point of the target function call, the tracking configuration is used to filter the variable information of the extracted relevant preset events, thereby reducing redundant information and optimizing tracking performance.

[0017] The tracking configuration includes rules for filtering by at least one of variable scope, data type, or specified key variable name.

[0018] Preferably,

[0019] The acquisition of the black-box pseudocode line includes:

[0020] The function call relationships are recorded by obtaining function call events and function return events through the tracking function;

[0021] Generate a function call tree based on the function call relationships and the entry function of the target function marked by the decorator;

[0022] A path fingerprint is generated based on the execution selection result of the condition judgment, and the path fingerprint represents an identified branch path; by comparing the set of path fingerprints generated when executing test cases corresponding to different inputs, the branch paths that have not yet been covered are identified, and all identified branch paths are obtained.

[0023] The function call tree and all identified branch paths are transformed into an AST-like visual structure to obtain the pseudocode lines of the black box.

[0024] Preferably, it further includes:

[0025] When the bytecode of the currently compiled black box changes due to a version update, the branch path tracing is retried to obtain the corresponding visualization structure of the updated black box.

[0026] Preferably, it further includes:

[0027] Generate a new version of the visual structure for the updated black-box code;

[0028] Compare the old and new versions of the visual structure view to identify structural differences;

[0029] And based on the structural differences, new supplementary test cases are generated.

[0030] Preferably, it further includes:

[0031] The decorator automatically verifies whether the tracked variable values ​​match the preset variable values ​​during test case execution, thereby verifying the intermediate logic.

[0032] According to a second aspect of the present invention, a black-box code analysis and test case generation apparatus is provided, the apparatus comprising:

[0033] Variable information acquisition module: used to register a tracing function through the preset sys.settrace interface, and to perform non-intrusive monitoring of the execution process of black-box code through the tracing function. The tracing function is called when a preset event is triggered, and is used to obtain variable information of the relevant preset event.

[0034] Variable trajectory acquisition module: used to establish a variable trajectory mapping relationship based on the changes in the variable values ​​of different variable names with the execution sequence and their corresponding bytecode offsets in the variable information of the relevant preset events;

[0035] Variable trajectory association module: Used to associate the variable trajectory mapping relationship with the pre-acquired black box pseudocode line through the instruction mapping function of the preset decompilation tool, so as to realize the understandable display of the variable trajectory;

[0036] Variable boundary value acquisition module: used to acquire variable boundary values ​​based on the distribution of variable values ​​in the variable trajectory and the branch paths in the pre-acquired black-box pseudocode lines;

[0037] Test case generation module: For branch paths that have not yet been covered, it reverse-engineers the combination of input variables that triggers the execution of the path based on the variable boundary values ​​and variable trajectories, and generates supplementary test cases based on the combination of input variables.

[0038] According to a third aspect of the present invention, a storage medium is provided, the storage medium storing a computer program, which, when executed by a host controller, implements the steps of the above-described method.

[0039] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects:

[0040] This application non-intrusively obtains the variable change trajectory and execution path within black-box code, making the previously invisible internal logic state transparent. This allows testers to accurately locate intermediate logic errors, solving the problem of traditional black-box testing's inability to observe internal states. Secondly, by systematically reconstructing the code's logical structure and execution path, this application can accurately identify uncovered logical branches and intelligently reverse-engineer and generate supplementary test cases, overcoming the shortcomings of traditional black-box testing, such as reliance on experience and incomplete coverage, significantly improving the depth and efficiency of testing. Thirdly, this application uses a system tracing interface for monitoring, without modifying the target code's bytecode, avoiding the potential impact of intrusive methods on program performance and stability. Combined with configurable tracing granularity control, it can effectively reduce the performance loss caused by redundant tracing.

[0041] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0042] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0043] Figure 1 This is a flowchart illustrating a black-box code analysis and test case generation method according to an exemplary embodiment;

[0044] Figure 2 This is a system schematic diagram of a black-box code analysis and test case generation apparatus according to another exemplary embodiment;

[0045] In the attached diagram: 1-Variable information acquisition module, 2-Variable trajectory acquisition module, 3-Variable trajectory association module, 4-Variable boundary value acquisition module, 5-Test case generation module. Detailed Implementation

[0046] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0047] Example 1

[0048] Figure 1 This is a flowchart illustrating a black-box code analysis and test case generation method according to an exemplary embodiment, such as... Figure 1 As shown, the method includes:

[0049] S1. Register a tracing function through the preset sys.settrace interface. The execution process of the black-box code is monitored non-intrusively through the tracing function. The tracing function is called when a preset event is triggered to obtain variable information of the relevant preset event.

[0050] S2, Based on the changes in the variable values ​​of different variable names with the execution sequence and their corresponding bytecode offsets in the variable information of the relevant preset events, establish a variable trajectory mapping relationship;

[0051] S3, by using the instruction mapping function of the preset decompilation tool, the variable trajectory mapping relationship is associated with the pre-acquired black box pseudocode line, so as to realize the understandable display of the variable trajectory;

[0052] S4, Obtain variable boundary values ​​based on the distribution of variable values ​​in the variable trajectory and the branch paths in the pre-acquired black-box pseudocode lines;

[0053] S5. For branch paths that have not yet been covered, the combination of input variables that triggers the execution of the path is deduced in reverse based on the variable boundary values ​​and variable trajectories, and supplementary test cases are generated based on the combination of input variables.

[0054] Understandably, compiled Python code (such as .pyc) cannot provide variable information through source code parsing. Traditional black-box testing can only observe input and output. This embodiment utilizes the low-level tracing capabilities of sys.settrace to achieve full-process tracing of variables from definition and assignment to destruction without modifying the target bytecode. Specifically, it includes:

[0055] By registering a tracing function using sys.settrace, when events such as call, line execution, and return are triggered during code execution, the tracing function extracts local variables (f_locals), global variables (f_globals), and variable type information from the current stack frame (frame object). Here, frame represents the currently executing stack frame, which contains context information of the executed code, such as the current function name and line number.

[0056] Local variables and their values: By accessing the collection of local variables in the execution stack frame object (e.g., frame.f_locals in CPython), you can obtain a dictionary containing all local variable names and their current values ​​within the current scope.

[0057] Global variables and their values: Similarly, the state of variables in the global scope can be obtained by accessing the collection of global variables in the execution stack frame object (e.g., frame.f_globals).

[0058] Bytecode offset: By accessing the current bytecode offset recorded in the execution stack frame object (e.g., frame.f_lasti), an integer that precisely points to the position of the bytecode instruction being executed within the function code object can be obtained.

[0059] Based on the changes in the variable values ​​of different variable names with the execution sequence and their corresponding bytecode offsets in the variable information of different preset events (call, line, return) obtained above, a variable trajectory mapping relationship is established. The variable trajectory mapping relationship is expressed as: variable name - value change - bytecode offset. For example, in the case of def func(a): b = a*2; return b, the association between the input value of a and the intermediate value of b at the bytecode level is traced.

[0060] To address the issue of .pyc files lacking source code line numbers, a mapping between bytecode offsets (f_lasti) and instructions from decompilation tools (such as uncompyle6) is used to associate variable traces with equivalent "pseudocode lines," thereby achieving an understandable display of variable traces.

[0061] It is worth emphasizing that obtaining the aforementioned "pseudocode line" includes:

[0062] By using the call and return events obtained from the trace functions registered with sys.settrace, the function call relationships (such as A() → B() → C()) are recorded, and combined with the entry functions marked by decorators (such as @trace_entry), a function call tree is generated;

[0063] Generate a "path fingerprint" (e.g., 0→1→0 represents the selection result of three branch judgments) from the execution result of conditional judgments (e.g., if x>0). Identify uncovered branches by comparing fingerprints from multiple test cases, specifically including:

[0064] During execution, whenever a conditional branch instruction (such as POP_JUMP_IF_FALSE) is encountered, the tracing function records the actual jump choice (i.e., whether the condition evaluation result is true or false). For a complete execution, all the conditional branches and their choices constitute a unique execution path. To identify this path, a unique path fingerprint can be generated based on the branch identifier and the sequence of selection results. For example, the bytecode offset of the branch and the Boolean selection result can be serialized and a hash value calculated. For the execution of target_func(200), since x>100 is true, a path fingerprint representing the "if branch" (called "Path-A") will be generated and stored in the set of covered paths;

[0065] By statically analyzing the pseudocode, all possible branch paths in the target function are identified. In the example of target_func, there are two paths: if and else. Then, all known paths are compared with the set of covered path fingerprints. After the initial execution of target_func(200), only "Path-A" is in the set. Therefore, it can be identified that the path corresponding to the else branch (called "Path-B") has not been covered.

[0066] Transform the function call tree and all identified branch paths into a visual structure similar to an AST (Abstract Syntax Tree), for example:

[0067] func_A()

[0068] ├─ Branch 1 (x>0) → Call func_B()

[0069] │└─ Loop (i=0→5)

[0070] └─ Branch 2 (x≤0) → Return directly;

[0071] The aforementioned visual structure is known as a "pseudocode line".

[0072] It is worth noting that when the bytecode of the compiled code changes due to a version update, the path tracing is automatically retried, and the visual structure view is updated, avoiding manual maintenance costs.

[0073] Analyze the distribution of values ​​in the variable trajectory (e.g., the value of x is concentrated in 0-100), combine it with the branch path (e.g., when x=100, a new branch is entered), and infer the boundary values ​​(e.g., 99, 100, 101).

[0074] For uncovered branch path fingerprints, the required combinations of input variables are derived in reverse (e.g., by solving for input conditions where x>0 and yisNone), generating supplementary use cases, specifically including:

[0075] To generate supplementary test cases for the uncovered path: Locate the key conditional branch leading to the uncovered path "Path-B", namely the else case of "if x>100", whose trigger condition is x<= 100. Utilize variable trajectory information, especially the type of the input variable x (e.g., inferring its numeric type from its initial value of 200), and use reverse derivation or a constraint solver to generate an input value that satisfies the condition x<= 100. A simple strategy is to choose a value some distance from the boundary, such as 50. Finally, based on the deduced input variable combination {x: 50}, generate a new, executable supplementary test case, namely target_func(50).

[0076] During test execution, decorators (such as @verify_case(expect_vars={"b": 20}) can be used to automatically verify whether the values ​​of the tracked variables meet expectations (without relying on the output), thus enabling the verification of intermediate logic.

[0077] When the compiled code is updated, the structural differences between the old and new versions are compared (such as the addition of the function func_C). The test cases that cover the original structure are automatically retained, and test cases for the new structure are added.

[0078] Example 2

[0079] As an optional implementation, this embodiment, based on Embodiment 1, provides a variant scheme for fine-grained control of the tracing process. This aims to address the problem that comprehensive tracing may lead to excessive performance overhead and information redundancy in large and complex applications. The technical solution of this embodiment supports flexible configuration of the tracing range and granularity.

[0080] In real-world software testing scenarios, the target black-box code might be a massive system containing hundreds or even thousands of functions and variables. Collecting all local and global variables for every line of code execution would generate a massive amount of data, consuming enormous storage space, severely slowing down the target program's execution, and placing a huge computational burden on subsequent processes. It's important to note that testers typically only care about a few key variables relevant to specific functionalities or defects.

[0081] To address this issue, this application introduces a tracing configuration and data filtering mechanism. As a preferred implementation, this embodiment provides a method for receiving tracing configurations through decorators. Decorators are syntactic sugar supported by many high-level programming languages ​​(such as Python), which can conveniently wrap or modify the behavior of functions or classes. Testers can use a predefined decorator in the test script that calls the target black-box code to declare the tracing requirements for this test.

[0082] For example, suppose the test script contains the following call:

[0083] # Code in the test script

[0084] from tracer_library import trace_vars

[0085] @trace_vars(scope="local", types=(int, str), names=["result", "status_code"])

[0086] def run_complex_test():

[0087] import target_module

[0088] # target_module.complex_func is a complex black-box function

[0089] target_module.complex_func(initial_data)

[0090] In this example, `trace_vars` is a configuration decorator provided in this embodiment. It receives three parameters, which together constitute a set of tracing configuration rules:

[0091] scope="local": This rule instructs the tracking function to only focus on variables in the local scope and ignore global variables.

[0092] types=(int, str): This rule further restricts the scope of local variables to only those of type integer or string.

[0093] names=["result", "status_code"]: This rule provides a more precise constraint, requiring that only variables with the name "result" or "status_code" be tracked.

[0094] Subsequently, when the `run_complex_test` function is executed, and subsequently calls the black-box function `target_module.complex_func`, the non-intrusive monitoring and data acquisition process described in Example 1 is still triggered. It should be noted that a filtering step is added in this example. After the tracing function extracts the complete set of local variables from the execution stack frame object each time a line of code execution is triggered, it does not immediately save all variable information; instead, it filters them according to preset configuration rules.

[0095] The specific filtering logic is as follows: For each variable (key-value pair) retrieved from the local variable set, a series of checks are performed, including scope checks, type checks, and name checks. For example, the scope configuration determines whether to process the local variable set or the global variable set; a type checking function (such as Python's isinstance) is used to determine whether the variable value's type belongs to one of the configured type tuples (int, str); and the variable's name is checked to see if it exists in the configured name list ["result", "status_code"].

[0096] A variable's name, current value, and corresponding bytecode offset will only be recorded when a variable satisfies all the configured rules simultaneously.

[0097] In this way, even if a complex black-box function contains a large number of temporary variables, loop counters, and objects of different types, this embodiment will only collect and analyze the key variables explicitly specified by the tester. This approach can significantly reduce the amount of data generated during tracing, reduce the consumption of storage and network resources, and significantly alleviate the performance burden on the target program, making the tracing process more lightweight. This fine-grained control capability enables the technical solution of this application to easily handle large-scale, performance-sensitive black-box application testing, achieving the goal of optimizing tracing performance while ensuring tracing accuracy.

[0098] Example 3

[0099] This embodiment, based on Embodiment 1, further provides a variant scheme that combines variable trajectory analysis to intelligently generate boundary value test cases. This scheme aims to address potential defects related to boundary condition handling that are difficult to detect using only branch coverage, such as the classic "one-off error".

[0100] In Example 1, the system generates supplementary test cases by identifying uncovered branch paths, with the primary goal of improving code structure coverage. However, even with 100% branch coverage, errors may still occur when the program handles specific boundary values. For example, logic that should handle x<= 100, if incorrectly implemented as x<100, will cause the program to exhibit unexpected behavior when the input is exactly 100. Traditional black-box testing methods rely on the tester's experience to design boundary value test cases, while this example provides an automated method for discovering and testing boundary values.

[0101] The following explanation will still use the target_func(x) function from Example 1 as an example. Assume that the system has executed two test cases in succession according to the process of Example 1: target_func(200) and target_func(50). At this time, the complete data of these two executions has been recorded, including their variable trajectories and the path fingerprints they cover ("Path-A" and "Path-B"), and the system knows that all branches have been covered.

[0102] At this point, the method in this embodiment will initiate a deeper analysis process. First, a boundary value analysis task is performed. The specific steps are as follows: A path switching event is identified, meaning that the program's execution path switched between two executions (from "Path-A" to "Path-B"). Then, the path switching event is associated with the code's logical structure to locate the key conditional branch that caused the path switching, i.e., the `if x>100` in the pseudocode. This conditional expression is analyzed, and the variable `x` is identified as the "key variable" determining the branch's direction. Next, the trajectory of the key variable `x` in the two executions is extracted and compared. It is observed that when the input value of `x` is 200, the `if` branch is executed; when the input value of `x` is 50, the `else` branch is executed. Based on this, the system performs a correlation analysis between the value of the key variable and the branch condition `x>100`, inferring that the path switching occurred when the value of `x` crossed the point of 100. Therefore, this value is identified and marked as an important "boundary value". It should be noted that the boundary value may be a constant explicitly stated in the code, or it may be an implicit value calculated from multiple variables. This method can effectively detect both.

[0103] After successfully inferring the boundary values, the task of generating boundary value test cases is executed. Based on boundary value analysis in software testing theory, a set of representative test inputs is automatically generated around the identified boundary values. A typical generation strategy includes:

[0104] Boundary point: Generate an input equal to the boundary value, i.e., x = 100. This use case is used to test whether the program behaves correctly at the boundary point itself.

[0105] Inside the boundary point: Generates a value that is immediately adjacent to the boundary and on the side where the condition is not met. For example, for integer types, x = 99.

[0106] Outside the boundary point: Generates a value that is immediately adjacent to the boundary and on the side where the condition is true. For example, for integer types, x = 101.

[0107] Based on this, three new supplementary test cases can be created: target_func(99), target_func(100), and target_func(101). These test cases are automatically added to the test set and can be scheduled for execution. By running these precise boundary value test cases and verifying their output or internal state, the test system can very effectively reveal program defects caused by incorrect boundary condition judgments (e.g., misusing > as >=, or vice versa).

[0108] This embodiment discovers implicit boundaries in the code by automatically analyzing the relationship between variable trajectories and path switching, and generates high-quality boundary value test cases accordingly. This greatly enhances the depth of testing and the ability to discover potential defects, making up for the shortcomings of testing methods that rely solely on branch coverage.

[0109] Example 4:

[0110] This embodiment provides a variant of generating a code structure view and using that view to intelligently adapt to regression testing. It aims to provide testers with an intuitive understanding of the black-box code logic and automate the regression testing maintenance work caused by code updates.

[0111] One aspect of this embodiment relates to the generation of a code logic structure view. In Embodiment 1, detailed information about the function call chain and conditional branch paths of the target black-box code was internally constructed through the analysis of runtime data. To transform this abstract data into a user-friendly form, this embodiment adds a structure view generation function. After completing the path analysis of the code, the reconstructed function call chain and all discovered logical branch paths are integrated to generate a visual code structure view. This view is conceptually similar to the abstract syntax tree of the source code, but it is obtained through reverse engineering of the dynamic execution of the black-box code.

[0112] In the structure view, nodes represent functions, conditional statements, and executed statements, while directed edges represent control flow. For example, for `target_func` in Example 1, the view displays a root node "target_func(x)", which connects to a diamond-shaped branch node "if (x>100)". This branch node then leads to two rectangular leaf nodes "y = x * 2" and "y = x + 10", with labels such as "True" and "False" indicating the path selection. The generated code structure view can be presented to users, providing testers with a clear blueprint of the internal workings of black-box code. This allows them to quickly understand the core logic even without the source code, which is very helpful for manually designing more complex test scenarios or understanding the root causes of errors.

[0113] Another aspect of this embodiment relates to intelligent adaptation of regression testing. Software products are continuously iterated and updated throughout their lifecycle. When the black-box code is updated, efficiently performing regression testing to ensure that old functions are not broken and to verify new functions is a significant challenge. This embodiment provides an automated solution.

[0114] Suppose the target_func function is modified by the developers and a new version is released, whose logic is equivalent to target_func_v2:

[0115] # New version of the target black-box function target_func_v2

[0116] def target_func_v2(x):

[0117] if x>200:

[0118] y = x * 3

[0119] elif x>100:

[0120] y = x * 2

[0121] else:

[0122] y = x + 10

[0123] return y

[0124] The new version adds an if x>200 branch and moves the original if x>100 logic to the elif branch.

[0125] Once testers receive the new version of the black-box code file, they will analyze it again using the system described in this application. First, the system will monitor, collect, and analyze the new version of the code, ultimately generating a new code logic structure view corresponding to `target_func_v2`. Then, it will compare the code structure views of the old and new versions, automatically identifying structural differences through tree comparison or graph comparison algorithms. In this example, the system will find that: a new top-level conditional branch `if (x>200)` and its corresponding execution body `y = x * 3` have been added; and the context of the original `if (x>100)` branch has changed, now becoming an `elif` branch.

[0126] Retain relevant test cases: System analysis revealed that the elif (x>100) and else branches in the new structure correspond to the logic of the if and else branches in the old structure, respectively. Therefore, the test cases previously generated to cover this logic (such as target_func(150) and target_func(50)) are still valid and need to be retained for regression testing.

[0127] Deprecated or updated test cases: If an old branch is completely removed, the corresponding test cases may be marked as deprecated.

[0128] Adding new feature test cases: The most crucial thing is that the system identifies the newly added if x>200 logical path as completely new and not covered by existing test cases, and then reverse-engineers the input that can trigger this path, such as generating a new test case target_func_v2(250).

[0129] Example 5:

[0130] Figure 2 This is a system schematic diagram of a black-box code analysis and test case generation apparatus according to another exemplary embodiment, the apparatus comprising:

[0131] Variable information acquisition module 1: Used to register a tracing function through the preset sys.settrace interface, and to perform non-intrusive monitoring of the execution process of black-box code through the tracing function. The tracing function is called when a preset event is triggered, and is used to obtain variable information of the relevant preset event.

[0132] Variable trajectory acquisition module 2: used to establish a variable trajectory mapping relationship based on the changes in the variable values ​​of different variable names with the execution sequence and their corresponding bytecode offsets in the variable information of the relevant preset events;

[0133] Variable trajectory association module 3: It is used to associate the variable trajectory mapping relationship with the pre-acquired black box pseudocode line through the instruction mapping function of the preset decompilation tool, so as to realize the understandable display of the variable trajectory;

[0134] Variable boundary value acquisition module 4: used to acquire variable boundary values ​​based on the distribution of variable values ​​in the variable trajectory and the branch paths in the pre-acquired black-box pseudocode lines;

[0135] Test case generation module 5: For branch paths that have not yet been covered, it reverse-engineers the combination of input variables that triggers the execution of the path based on the variable boundary values ​​and variable trajectories, and generates supplementary test cases based on the combination of input variables.

[0136] Example 6:

[0137] This embodiment provides a storage medium storing a computer program, which, when executed by a host controller, implements the various steps in the above method.

[0138] It is understood that the storage medium mentioned above can be a read-only memory, a hard disk, or an optical disk, etc.

[0139] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0140] It should be noted that in the description of this invention, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means at least two.

[0141] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0142] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0143] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0144] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0145] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0146] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0147] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for black-box code analysis and test case generation, characterized in that, The method includes: A tracing function is registered through the preset sys.settrace interface. The execution process of the black-box code is monitored non-intrusively through the tracing function. The tracing function is called when a preset event is triggered to obtain variable information of the relevant preset event. The tracking function is invoked when a preset event is triggered, and the variable information of the relevant preset event is obtained, including: When a preset event is triggered, the tracing function extracts runtime context data from the current execution stack frame. The context data includes local variables, global variables, and variable type information. Based on the changes in the variable values ​​of different variable names with the execution sequence and their corresponding bytecode offsets in the variable information of the relevant preset events, a variable trajectory mapping relationship is established; By using the instruction mapping function of the preset decompilation tool, the variable trajectory mapping relationship is associated with the pre-acquired black box pseudocode line, so as to realize the understandable display of the variable trajectory; The acquisition of the black-box pseudocode line includes: The function call relationships are recorded by obtaining function call events and function return events through the tracking function; Generate a function call tree based on the function call relationships and the entry function of the target function marked by the decorator; A path fingerprint is generated based on the execution selection result of the condition judgment, and the path fingerprint represents an identified branch path; by comparing the set of path fingerprints generated when executing test cases corresponding to different inputs, the branch paths that have not yet been covered are identified, and all identified branch paths are obtained. The function call tree and all identified branch paths are transformed into an AST-like visual structure to obtain the pseudocode lines of the black box; Obtain the variable boundary values ​​based on the distribution of variable values ​​in the variable trajectory and the branch paths in the pre-acquired black-box pseudocode lines; For branch paths that have not yet been covered, the combination of input variables that triggers the execution of that path is deduced in reverse based on the variable boundary values ​​and variable trajectories, and supplementary test cases are generated based on the combination of input variables.

2. The method according to claim 1, characterized in that, The preset events include at least: function call events, code line execution events, and function return events.

3. The method according to claim 2, characterized in that, Also includes: By setting a decorator at the entry point of the target function call, the tracking configuration is used to filter the variable information of the extracted relevant preset events, thereby reducing redundant information and optimizing tracking performance. The tracking configuration includes rules for filtering by at least one of variable scope, data type, or specified key variable name.

4. The method according to claim 3, characterized in that, Also includes: When the bytecode of the currently compiled black box changes due to a version update, the branch path tracing is retried to obtain the corresponding visualization structure of the updated black box.

5. The method according to claim 4, characterized in that, Also includes: Generate a new version of the visual structure for the updated black-box code; Compare the old and new versions of the visual structure view to identify structural differences; And based on the structural differences, new supplementary test cases are generated.

6. The method according to claim 5, characterized in that, Also includes: The decorator automatically verifies whether the tracked variable values ​​match the preset variable values ​​during test case execution, thereby verifying the intermediate logic.

7. A black-box code analysis and test case generation device, characterized in that, The device includes: Variable information acquisition module: used to register a tracing function through the preset sys.settrace interface, and to perform non-intrusive monitoring of the execution process of black-box code through the tracing function. The tracing function is called when a preset event is triggered, and is used to obtain variable information of the relevant preset event. The tracking function is invoked when a preset event is triggered, and the variable information of the relevant preset event is obtained, including: When a preset event is triggered, the tracing function extracts runtime context data from the current execution stack frame. The context data includes local variables, global variables, and variable type information. Variable trajectory acquisition module: used to establish a variable trajectory mapping relationship based on the changes in the variable values ​​of different variable names with the execution sequence and their corresponding bytecode offsets in the variable information of the relevant preset events; Variable trajectory association module: Used to associate the variable trajectory mapping relationship with the pre-acquired black box pseudocode line through the instruction mapping function of the preset decompilation tool, so as to realize the understandable display of the variable trajectory; The acquisition of the black-box pseudocode line includes: The function call relationships are recorded by obtaining function call events and function return events through the tracking function; Generate a function call tree based on the function call relationships and the entry function of the target function marked by the decorator; A path fingerprint is generated based on the execution selection result of the condition judgment, and the path fingerprint represents an identified branch path; by comparing the set of path fingerprints generated when executing test cases corresponding to different inputs, the branch paths that have not yet been covered are identified, and all identified branch paths are obtained. The function call tree and all identified branch paths are transformed into an AST-like visual structure to obtain the pseudocode lines of the black box; Variable boundary value acquisition module: used to acquire variable boundary values ​​based on the distribution of variable values ​​in the variable trajectory and the branch paths in the pre-acquired black-box pseudocode lines; Test case generation module: For branch paths that have not yet been covered, it reverse-engineers the combination of input variables that triggers the execution of the path based on the variable boundary values ​​and variable trajectories, and generates supplementary test cases based on the combination of input variables.

8. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by the main controller, implements each step of the black-box code analysis and test case generation method as described in any one of claims 1-6.

Images