A code analysis tool management method, service device, equipment and medium

Abstract

The application provides a management method of a code analysis tool, the code analysis tool carrying a code analysis module. The method comprises the following steps: obtaining at least one user repair instance according to the repair behavior of a user of the code analysis tool on a defective code; generating a test prediction according to the user repair instance and at least one system repair instance generated by the code analysis tool on the defective code; and processing the code analysis module according to the test prediction, so as to realize the management of the code analysis tool, thereby solving the problem of high labor cost and time cost, and having high usability.

Description

Technical Field

[0001] This application relates to the field of computers, and more particularly to a method, apparatus, device, and medium for managing code analysis tools. Background Technology

[0002] During software development, it is often necessary to detect defects in the software code. Considering the cost of detection, the industry has provided a code analysis tool. This tool includes a code analysis module, which, when run, enables automated code defect detection.

[0003] The effectiveness of code analysis modules directly determines the quality of code defect detection performed by code analysis tools. Effectiveness can be measured by at least one metric: false positives ratio (FPR) and false negatives ratio (FNR). To ensure the proper functioning of code analysis tools, the effectiveness of code analysis modules generally needs to be monitored. Specifically, when the effectiveness of a code analysis module is low, it is optimized to improve its effectiveness. Furthermore, when a code analysis module is optimized, its effectiveness also needs to be monitored, i.e., whether the optimized module introduces new effectiveness issues.

[0004] Currently, the industry has proposed several methods to monitor and improve the effectiveness of code analysis modules. However, these existing methods require significant manpower and time costs, making them impractical. Therefore, providing a management method for code analysis tools to ensure the effectiveness of their code analysis modules has become a pressing issue. Summary of the Invention

[0005] This application provides a management method for code analysis tools, solving the problem of high manpower and time costs associated with managing such tools, and is highly feasible. This application also provides corresponding devices, computer clusters, computer-readable storage media, and computer program products.

[0006] Firstly, this application provides a management method for code analysis tools, which can be applied to a service device. The service device is a device that provides management services for the code analysis tools. The service device, together with user devices (devices of users of the code analysis tools) and development devices (devices of developers of the code analysis tools), constitute a management system for the code analysis tools.

[0007] The service device deploys a code analysis tool, which carries a code analysis module. The service device can use this module to analyze the code and identify defective code. In specific implementations, the service device can obtain at least one user repair instance based on the user's actions in fixing the defective code. Then, based on the user repair instance and at least one system repair instance generated by the code analysis tool, it generates a test oracle. Finally, it processes the code analysis module according to the test oracle to manage the code analysis tool.

[0008] This method automatically generates test oracles and manages code analysis tools based on these oracles, avoiding the manpower and time costs associated with developers manually writing test instances. Furthermore, it eliminates the need for developers to communicate with each user individually to obtain false positive codes, saving communication costs, improving the management efficiency of code analysis tools, and demonstrating high usability.

[0009] In conjunction with the first aspect, in a first possible implementation of the first aspect, the service device can determine at least one of a correct repair instance, a missed repair instance, and a false positive repair instance based on the association between user repair instances and system repair instances, and then generate a test oracle based on at least one of the correct repair instance, the missed repair instance, and the false positive repair instance. Correspondingly, the test oracle includes at least one of a correct test oracle, a missed test oracle, and a false positive test oracle.

[0010] In this way, user intent can be intelligently extracted through code analysis tools during user interaction, and test predictions can be automatically generated based on user intent, eliminating the need for developers to manually write test cases.

[0011] In conjunction with the first implementation of the first aspect, in the second implementation of the first aspect, the test oracle specifically includes a correct test oracle. Based on this, the service device can perform regression testing on the code analysis module according to the correct test oracle to determine the effectiveness of the code analysis module, thereby achieving the monitoring of the code analysis module.

[0012] In conjunction with the first or second implementation of the first aspect, in the third implementation of the first aspect, the test oracle specifically includes a false negative test oracle and / or a false positive test oracle. Based on this, the service device can update the code analysis module according to the false negative test oracle and / or the false positive test oracle, thereby improving the effectiveness of the code analysis module.

[0013] In the fourth implementation of the first aspect, which combines any one of the first to third implementations of the first aspect, the user's behavior of repairing defective code specifically refers to the user's secondary repair behavior on the code after it has been repaired by the code analysis tool.

[0014] At this time, the service device can determine the correct repair instance, the missed repair instance, and / or the false alarm repair instance in the following ways: when the system repair instance fails to be associated with the user repair instance, the system repair instance is determined to be the correct repair instance; when the system repair instance is successfully associated with the user repair instance, the system repair instance is determined to be the false alarm repair instance; when the user repair instance fails to be associated with the system repair instance, the user repair instance is determined to be the missed repair instance.

[0015] This process can be completed without the user's awareness, reducing user operations and improving the automation level of the code analysis tool's management system.

[0016] In conjunction with the first aspect or any of the first to fourth implementations of the first aspect, in the fifth implementation of the first aspect, the test oracle records an identifier of the defect code, and the identifier of the defect code is used to index the defect code and the context of the defect code.

[0017] Because the defect code identifiers in test oracles can index both the defect code and its context, this avoids situations where test oracles are defined solely in the order of analysis within test cases, leading to test failures due to the inability to retrieve the corresponding code in certain test scenarios. Furthermore, it enables cross-file analysis of the code to be analyzed, improving the accuracy of defect code detection.

[0018] Secondly, this application provides a service apparatus, the apparatus comprising: a communication unit, a generation unit, a management unit, and an analysis unit; wherein the analysis unit includes a code analysis tool, and the code analysis tool includes a code analysis module; The communication unit is used to obtain at least one user repair instance based on the user's repair behavior of the defective code in the code analysis tool. The generation unit is used to generate a test prediction based on at least one system repair instance generated by the user repair instance and the code analysis tool used by the analysis unit to repair the defective code; The management unit is also used to process the code analysis module according to the test prediction in order to manage the code analysis tool.

[0019] In conjunction with the second aspect, in a first possible implementation of the second aspect, the generating unit is specifically used for: Based on the association between the user repair instance and the system repair instance, at least one of the following is determined: a correct repair instance, a missed repair instance, and a false alarm repair instance; A test oracle is generated based on at least one of the correct repair instance, the missed repair instance, and the false positive repair instance, wherein the test oracle includes at least one of the correct test oracle, the missed repair instance, and the false positive test oracle.

[0020] In conjunction with the first implementation of the second aspect, in the second implementation of the second aspect, the test oracle includes a correct test oracle; The management unit is specifically used for: Regression testing is performed on the code analysis module based on the correct test predictions to determine the effectiveness of the code analysis module.

[0021] In conjunction with the first or second implementation of the second aspect, in the third implementation of the second aspect, the test oracle includes a false negative test oracle and / or a false positive test oracle; The management unit is specifically used for: The code analysis module is updated based on the false negative test prediction and / or false positive test prediction.

[0022] In combination with any of the first to third implementations of the second aspect, in the fourth implementation of the second aspect, the user's repair behavior on defective code includes the user's secondary repair behavior on the code after it has been repaired by the code analysis tool; The generation unit is specifically used for: When the system repair instance fails to associate with the user repair instance, the system repair instance is determined to be the correct repair instance. When the system repair instance is successfully associated with the user repair instance, the system repair instance is determined to be a false alarm repair instance; When the association between the user repair instance and the system repair instance fails, the user repair instance is determined to be a missed repair instance.

[0023] In a fifth implementation of the second aspect, in conjunction with the second aspect or any of the first to fourth implementations of the second aspect, the test oracle records an identifier of the defect code, and the identifier of the defect code is used to index the defect code and the context of the defect code.

[0024] Thirdly, this application provides a computer cluster comprising at least one computer, each computer including a processor and a memory. The processor of the at least one computer is used to execute instructions stored in the memory of the at least one computer to perform a management method for code analysis tools as described in the first aspect or any implementation thereof.

[0025] Fourthly, this application provides a computer-readable storage medium storing instructions that, when run on a computer cluster, cause the computer cluster to perform the method described in the first aspect or any implementation thereof.

[0026] Fifthly, this application provides a computer program product containing instructions that, when run on a computer cluster, causes the computer cluster to perform the method described in the first aspect or any implementation thereof. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the management system of the code analysis tool in the embodiments of this application; Figure 2 This is a flowchart illustrating the management method of the code analysis tool in an embodiment of this application; Figure 3 This is a schematic diagram illustrating how a user repairs defective code in an embodiment of this application; Figure 4 This is a schematic diagram illustrating the generation of test oracles in an embodiment of this application; Figure 5 This is a schematic diagram illustrating the selection of a repair instance to send the corresponding test oracle in an embodiment of this application; Figure 6 This is a schematic diagram illustrating a newly added example of missed detection repair in the embodiments of this application; Figure 7 This is a schematic diagram of the list of optimized repair capabilities in the embodiments of this application; Figure 8 This is a schematic diagram illustrating the optimization of repair capabilities in an embodiment of this application; Figure 9 This is a schematic diagram of the structure of the service device in the embodiments of this application; Figure 10 This is a schematic diagram of the structure of a computer cluster in an embodiment of this application. Detailed Implementation

[0028] In software development, code analysis tools are often used to analyze the code to detect and fix bugs. These tools typically contain at least one code analysis module, which includes code analysis rules. Each module can detect at least one type of bug using these rules.

[0029] To meet users' demands for the reliability of code analysis tools, the effectiveness of the code analysis modules within these tools needs to be monitored. Considering that the code analysis modules are primarily used to detect defective code, their effectiveness can be measured from at least one dimension, such as precision and completeness.

[0030] The effectiveness of the code analysis module can be measured by the false positive ratio (FPR) from the precision dimension, and by the false negative ratio (FNR) from the recall dimension.

[0031] The false positive rate (FPR) is the ratio of the number of non-defect code identified as defect code by the code analysis module (referred to as false positive, FP in this application) to the sum of FP and the actual defect code detected by the code analysis module (referred to as true negative, TN in this application).

[0032] The false negative rate refers to the ratio of the code analysis module identifying real defective code as non-defective code (referred to as false negative, or FN in this application) to the sum of FN and non-defective code detected by the code analysis module (referred to as true positive, or TP in this application).

[0033] Because code implementation scenarios are complex and varied, it's difficult for code analysis tool developers to achieve a low false positive and low false negative rate code analysis module in a single development effort. Therefore, developers can first release a version of the code analysis module and then optimize it based on user feedback.

[0034] It should be noted that optimizing a code analysis module may introduce new issues that could affect its effectiveness. Therefore, developers of code analysis tools can also perform regression testing on the code analysis modules within their tools to verify their effectiveness.

[0035] Currently, the industry has proposed several methods and tools for automatically detecting the effectiveness of code analysis modules, as well as methods and tools for improving the effectiveness of code analysis modules. Tools for automatically detecting the effectiveness of code analysis modules can include llvm-lit. Tools for improving the effectiveness of code analysis modules can include tricoder.

[0036] llvm-lit includes pre-written test cases by developers. A test case describes a testing task for a software product and includes at least test inputs and testoracles. Test inputs can include test code, and testoracles can be understood as the expected test results, used to check the correctness of the test code execution. llvm-lit calls the code analysis module to analyze the test cases, comparing the analysis results with the expected test results. If they match, the test case passes, thus enabling regression testing. This regression testing method ensures the effectiveness of the code analysis module within the test case coverage scenarios, but it requires developers to manually design and add test cases, resulting in significant time and manpower costs.

[0037] Tricoder integrates multiple analyzers, each capable of detecting and fixing defect codes of different types. Tricoder presents the detection results for each task execution to the user, who can then confirm and provide feedback. Tricoder periodically calculates the false positive rate for each analyzer. When the false positive rate exceeds a preset value, it notifies the developer of the corresponding analyzer. The developer then needs to contact the user to obtain the false positive code to assist in optimizing the code analysis module. Because the developer's development environment and the user's usage environment are separate, developers must obtain the false positive code by communicating with each user individually. This reduces the efficiency of optimizing the code analysis module and increases optimization costs.

[0038] Based on this, this application provides a management method for code analysis tools. This method automatically generates test precepts by utilizing user-defined repair instances (determined by users' actions to fix defective code) and system-defined repair instances (determined by the code analysis tool itself). The code analysis module is then processed according to these test precepts, thereby achieving management of the code analysis tool. On one hand, developers no longer need to manually design and add test cases, saving time and manpower costs. On the other hand, developers no longer need to obtain false positive codes through multiple communications with each user; instead, they can directly optimize the code analysis module based on the automatically generated test precepts, further improving efficiency, reducing optimization costs, and demonstrating high usability.

[0039] The embodiments of this application will now be described with reference to the accompanying drawings.

[0040] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application.

[0041] First, see Figure 1 The diagram shows the structure of a management system for a code analysis tool. This management system includes a development device 102, a service device 104, and at least one user device 106. The computer deploying the development device 102 and the user device 106 can be a desktop computer, laptop computer, smartphone, or server, etc. The computer deploying the service device 104 can be a server, such as a cloud server, or a server cluster. The development device 102 can also be deployed on the same computer as the service device 104.

[0042] The code analysis tool includes a code analysis module. Users of the code analysis tool can trigger code analysis operations using the graphical user interface (GUI) or command-line interface (CLI) of device 106. In response to the aforementioned code analysis operations, service device 104 analyzes the code to be analyzed using the code analysis module within the code analysis tool to obtain defective code.

[0043] Service device 104 can send defective code to user device 106. Users of the code analysis tool can perform repair actions on the defective code displayed by user device 106. User device 106 obtains at least one user repair instance (manual change instance) based on the user's repair actions. Furthermore, service device 104 can also repair defective code detected by the code analysis tool, generating at least one system repair instance (system change instance).

[0044] Service device 104 can generate test predictions based on the aforementioned user repair instances and system repair instances, and send the test predictions to development device 102. Development device 102 receives the processing results of the code analysis module by the developers based on the test predictions, and sends the processing results to service device 104. Service device 104 processes the code analysis module based on the processing results, thereby realizing the management of code analysis tools.

[0045] It should be noted that, Figure 1This implementation uses a code analysis tool to host a code analysis module, and each code analysis module corresponds to a developer who provides examples and explanations. In practical applications, each code analysis module can correspond to one or more developers, and the code analysis tool can host one or more code analysis modules; this embodiment does not impose any limitations on this.

[0046] Furthermore, development device 102 can also trigger code analysis operations. In this case, development device 102 is equivalent to user device 106, and the user of development device 102 can be considered a user. Similarly, user device 106 can also receive test oracles and process the code analysis module based on the test oracles. In this case, user device 106 is equivalent to development device 102, and the user of user device 106 can also be considered a developer.

[0047] To make the technical solution of this application clearer and easier to understand, the management method of the code analysis tool in this application will be introduced from the perspective of the interaction between the development device 102, the service device 104 and the user device 106.

[0048] See Figure 2 The flowchart shown illustrates a management method for code analysis tools. This method is applied to a management system for code analysis tools, which includes a development device 102, a service device 104, and a user device 106. The method specifically includes the following steps: S202: The user of the code analysis tool triggers the code analysis operation by using device 106.

[0049] The device 106 provides at least one of a graphical user interface or a command-line interface. The graphical user interface includes code analysis controls. Users of the code analysis tool can trigger code analysis operations on specified code through these controls. The command-line interface includes command input controls. Users of the code analysis tool can also input code analysis commands through these controls to trigger code analysis operations on specified code.

[0050] It should also be noted that in some cases, code analysis operations can be automatically triggered when preset conditions are met. For example, the code analysis operation can be automatically triggered before the code goes live, etc., and this embodiment does not limit this.

[0051] S204: In response to the code analysis operation, the device 106 sends a code analysis request to the service device 104.

[0052] The code analysis request is used to request the service device 104 to analyze specified code. In a specific implementation, the code analysis request may include an identifier of the specified code, so that the service device 104 can obtain the specified code based on the identifier and perform code analysis.

[0053] The identifier for the specified code may include the name of the code file to which the specified code belongs. In some cases, to facilitate the service device 104 in quickly obtaining the specified code, the identifier for the specified code may also include the path to the code file to which the specified code belongs. Considering the possibility of analyzing partial code segments within a code file, the identifier for the specified code may also include the location of the specified code within its respective code file. The location can be identified using line numbers, etc.

[0054] In some possible implementations, the code analysis request may also include specified code, so that the service device 104 can directly obtain the specified code from the code analysis request for code analysis. This reduces the number of data transfers between the user device 106 and the service device 104, avoiding unnecessary transmission resource overhead.

[0055] S206: Service device 104 uses the code analysis module in the code analysis tool to analyze the code to be analyzed and obtain defect code.

[0056] After obtaining the specified code, the service device 104 uses the specified code as the code to be analyzed. Then, the service device 104 uses at least one code analysis module in the code analysis tool to analyze the code to be analyzed, and obtain defect codes of at least one defect type.

[0057] Each code analysis module in the code analysis tool can detect defective code of at least one defect type. The defect types detected by each module are different. The developers of each module determine the defect types they detect when developing the module. In one example, defect types might include comment space defects, pointer alignment defects, or header file defects. A comment space defect refers to comments without spaces, and a pointer alignment defect refers to pointers without spaces. "The header file defect type refers to the inclusion of unnecessary header files, where spaces are not provided on either side of the variable name or pointer."

[0058] S208: Service device 104 uses the code analysis module to automatically repair defective codes and generate at least one system repair instance.

[0059] In this application, the code analysis module is also used to automatically repair detected defective code. The service device 104 can use a code analysis tool to call the code analysis module to repair the defective code. Then, based on the code analysis tool's repair behavior for the defective code, the service device 104 generates at least one system repair instance through behavior decomposition.

[0060] To facilitate understanding, a specific example is provided below. In this example, when the service device 104 uses a code analysis module to detect defect codes of a certain defect type, assuming that three defect codes of that defect type are detected, the service device 104 also automatically repairs these three defect codes. The code fragments of each defect code before and after automatic repair can generate a system repair instance, thus obtaining three system repair instances.

[0061] S210: Service device 104 sends a defect code to user device 106.

[0062] In practical applications, the service device 104 can send defect codes or system repair instances generated based on defect codes to the user device 106, so that the user device 106 can display the defect codes or system repair instances generated based on defect codes. Users of the code analysis tool can directly perform repairs based on the defect codes, or they can perform secondary repairs based on the repairs performed by the code analysis tool.

[0063] To facilitate understanding, the following example uses secondary repair as an illustration, along with a visual interface diagram. Figure 3 As shown, the device 106 generates a user processing window based on a system repair instance. The left side of this user processing window includes a defect file view, which displays a list of defective files. The right side of the user processing window includes a pre-repair view, a post-repair view, and a defect alarm view. The pre-repair view displays the defect code before repair, the post-repair view displays the repaired code, and the defect alarm view displays at least one alarm description. This alarm description may include one or more of the following: defect type and corresponding description information.

[0064] Users of the code analysis tool can select any defect file from the defect file list through the user processing window. The right side of the window then displays the defect code, the repaired code from the selected defect file, and the corresponding alarm description information in the pre-repair view, post-repair view, and defect alarm view. Users can determine whether to perform a secondary repair based on each defect code, the repaired code, and the alarm description. If so, they can edit the repaired code in the post-repair view to perform the secondary repair.

[0065] In some cases, such as when the secondary repair involves significant changes, users of code analysis tools can revert the repaired code in the post-repair view to the code before the repair, and then perform a secondary repair based on the original defective code to obtain the user-repaired code.

[0066] It should be noted that when the service device 104 sends the defect code to the user device 106, S208 and S210 can be executed in parallel or sequentially according to the set order.

[0067] S212: Using device 106, at least one user repair instance is obtained based on the user's repair behavior of the defective code in the code analysis tool.

[0068] As described in S210, a user's action to fix defective code can be either a fix for the defective code before automatic fixing or a secondary fix for the code after it has been fixed by the code analysis tool. Therefore, a user-fixed instance can include the defective code and the code after the user's fix. Of course, when a user performs a secondary fix on the code after it has been automatically fixed by the code analysis tool, a user-fixed instance can also include the defective code, the code after the automatic fix, and the user-fixed code.

[0069] Device 106 can obtain at least one user repair instance by splitting the user's repair behavior on defective code using the code analysis tool. The splitting process can refer to the specific implementation of the code analysis tool's repair behavior on defective code using service device 104, and will not be repeated here.

[0070] S214: The device 106 sends a user repair instance to the service device 104.

[0071] In practical applications, considering code security, the user device 106 can first send the identifier of the user repair instance to the service device 104. This identifier may include the storage path and name of the user repair instance. The service device 104 can then retrieve the user repair instance from the user device 106 based on this identifier and load it into volatile storage devices such as memory. This avoids the security risks caused by the user repair instance being leaked in the service device 104.

[0072] In some possible implementations, device 106 can collect the code analysis tool user's code repair behavior and send it to service device 104. Service device 104 then breaks down the code repair behavior to obtain at least one user repair instance.

[0073] It should be noted that, considering business needs, the user device 106 also supports a submission confirmation function. Specifically, before sending the user repair instance to the service device 104, the user device 106 displays a list of user repair instances in a user processing window. The user can select at least one user repair instance from the user repair list and then send the user-selected user repair instance to the service device.

[0074] S216: Service device 104 generates a test prediction based on system repair instances and user repair instances.

[0075] A test oracle includes the expected test results for the test code. Specifically, the test code is the code to be analyzed as described in S206. In practical applications, the expected test results may be included in a test task list. The test task list describes at least one test task, and each test task corresponds to one test oracle. Each test task can be described by the following fields: test task identifier (id), defect description information (analysis_warnings), and repair description information (related_code_ranges). Correspondingly, the expected test results may include the aforementioned defect description information and repair description information.

[0076] The test task identifier can be any unique identifier used to distinguish test tasks.

[0077] `analysis_warnings` represents the analysis results from code analysis tools. These results can be at least one data pair consisting of the location of defective code in the test code and a defect warning description. The location of the defective code can include the path and name of the code file to which it belongs (`main_file`) and the line number of the defective code within that file (`main_line`). The defect warning description (`warning_description`) can include the defect type corresponding to the defective code.

[0078] `related_code_ranges` is used to characterize the expected fix for the aforementioned defective code. This fix can include the location of the sugegested changes in the fix file. In some cases, the fix can also be at least one data pair consisting of the sugegested changes in the fix file and their location. The location of the sugegested changes can include the path and name of the fix file to which the changes belong (changed file), the start line number of the change (start_line), and the end line number of the change (end_line).

[0079] In this embodiment, the test oracle is associated with the line number of the test code and the line number of the repaired code in the repair file. This avoids the situation where the test oracle is defined in the test case only according to the analysis order, which may lead to the inability to obtain the corresponding code in some test scenarios and thus cause test failure.

[0080] Furthermore, when the code file to which the defective code belongs also contains related files (hereinafter referred to as associated files), such as files with a referencing relationship, the analysis results can also include the paths and names of the aforementioned associated files. That is, the analysis results can be at least one data pair consisting of the location of the defective code in the test code, the path and name of the associated file, and the defect alarm description information.

[0081] The location of the defective code in the test code, along with the path and name of its associated file, can serve as an identifier for the defective code. This identifier can be used to index the defective code and its context. Specifically, the context refers to code segments that are related to the defective code, such as a code segment formed by n lines of code before and after the defective code. Of course, if the file containing the defective code has associated files, the context also includes code segments within those associated files. Based on this, the test oracle can also perform cross-file analysis, improving the accuracy of defect detection.

[0082] In practical applications, such as Figure 4 As shown, the service device 104 first determines at least one of the correct repair instance, the missed repair instance, and the false alarm repair instance based on the association between the system repair instance and the user repair instance. Then, the service device 104 can generate a correct test prediction based on the correct repair instance, a missed test prediction based on the missed repair instance, and a false alarm test prediction based on the false alarm repair instance.

[0083] Specifically, if a user's code repair action is a secondary repair action on code already fixed by a code analysis tool, the service device 104 can determine that the system repair instance is a correct repair instance when the association between the system repair instance and the user repair instance fails. If the association between the system repair instance and the user repair instance is successful, the system repair instance is determined to be a false positive repair instance. If the association between the user repair instance and the system repair instance fails, the user repair instance is determined to be a missed repair instance.

[0084] Correspondingly, each test task in the test task list can also be described by a tag field, which indicates the type of test oracle it corresponds to. In one example, a tag value of 0 indicates that the test oracle is a correct test oracle, a tag value of 1 indicates that the test oracle is a false positive test oracle, and a tag value of 2 indicates that the test oracle is a false negative test oracle.

[0085] S218: Service device 104 sends a test oracle to development device 102.

[0086] The service device 104 can directly send test oracles to the development device 102. In some possible implementations, the service device 104 may first send the identifier of the test oracle to the development device 102. The identifier of the test oracle may be the storage path and name of the test oracle. In this way, the development device 102 can obtain the test oracle from the service device 104 according to the identifier of the test oracle.

[0087] It should be noted that, considering business needs, service device 104 also supports user selection of test oracles and sending the selected test oracles to development device 102. Specifically, after the user of the code analysis tool exits the defect code repair through device 106, device 106 can display a list of repair instances through the user processing window. Figure 5 As shown, the user processing window can display at least one of the following categories: a list of false positive repair instances, a list of false negative repair instances, and a list of correct repair instances. The user of the code analysis tool can select a repair instance from one of the lists displayed in the user processing window. In this way, the service device 104 can send the test oracle corresponding to the repair instance selected by the user to the development device 102.

[0088] For both false positive and corrected repair instances, users of the code analysis tool can directly submit their repair requests via the submission control after selecting the instance. Figure 5 The "Confirm Contribution to Repair Instance" control triggers a submission operation, and the service device 104 responds to the submission operation by sending the corresponding test oracle to the development device 102.

[0089] For a missed repair instance, the user processing window of device 106 will redirect to display a defect alarm view. The user of the code analysis tool selects the alarm description information corresponding to the missed repair instance in the defect alarm view. Then, service device 104 sends a test oracle generated based on the repair instance described in the alarm description information to development device 102.

[0090] It's important to note that code analysis tools may miss reports in two ways: First, the tool analyzes the code to be analyzed and identifies the location of the defective code, but fails to capture all corresponding code snippets. Second, the tool analyzes the code to be analyzed but fails to identify either the location of the defective code or the corresponding code snippets. For the first situation, users of the code analysis tool can directly select the corresponding alarm description in the defect alarm view. For the second situation, users can also manually add alarm descriptions to select the alarm description and trigger a submission.

[0091] Specifically, users of the code analysis tool can trigger the addition of alarm description information by adding a new control, and the device 106 can display the alarm description information input control via a pop-up window. For example... Figure 6 As shown, users of the code analysis tool can input the defect type using the defect type input control and the name of the defect file to which the defect code belongs using the file name input control. In some possible implementations, the file name input control allows users to enter the defect file name by browsing folders. Furthermore, the device 106 can also display the code in the defect file, allowing users to select the starting line number of the defect code. After completing the input of the alarm description information, users of the code analysis tool can click the submit control. Figure 6 The "Confirm Report" control in the middle triggers the submission operation. In response to the operation, the service device 104 sends the test prediction generated by the repair instance described by the newly added alarm description information to the development device 102.

[0092] Considering that test oracles generated from multiple user-reported fix instances may be duplicated, service device 104 can filter duplicate test oracles before sending them to development device 102. This reduces transmission resource overhead and also reduces the workload of code analysis tool developers, improving processing efficiency.

[0093] S220: Development device 102 receives the processing results of the code analysis module by the developer of the code analysis tool based on test predictions.

[0094] The code analysis tool developer's processing results for the code analysis module based on test oracles include at least one of regression test results and update results. When the test oracles include correct test oracles, the processing results include regression test results. When the test oracles include at least one of false positive test oracles and false negative test oracles, the processing results include update results. The update results may include at least one of the results of optimizing for false negatives in the code analysis module and the results of optimizing for false positives in the code analysis module.

[0095] Specifically, the development device 102 can generate a developer processing window based on test premonitions, through which the code analysis tool developer can process the code analysis module. When the test premonitions include correct test premonitions, the code analysis tool developer can trigger a regression test operation through the developer processing window. The development device 102 then invokes the regression testing tool to perform regression testing on the code analysis module using the correct test premonitions, determining the effectiveness of the code analysis module. When the test premonitions include false negative test premonitions and / or false positive test premonitions, the code analysis tool developer can trigger an update operation through the developer processing window, updating the code analysis module according to the false negative test premonitions and / or false positive test premonitions, thereby optimizing the false negative and / or false positive situations of the code analysis module.

[0096] In specific implementation, such as Figure 7 As shown, the developer processing window displays a list of remediation capability optimizations, which can be generated from all false positive and false negative test predictions. Furthermore, the developer processing window also supports filtering the remediation capability optimization list by defect type and remediation status, and displays the filtered remediation capability optimization list. Each remediation capability optimization entry in the list corresponds to one test predict.

[0097] Developers of code analysis tools can click on any fix optimization item in the fix optimization list, and the developer processing window will display detailed information about the corresponding test predictions. For example... Figure 8As shown, the developer processing window displays the defect type, fix status, defect code, code fixed by the code analysis tool, and expected fix code (i.e., the sugegested changes mentioned in S216). The developer of the code analysis tool can enter the address of the code repository where the code analysis module is located in the code repository address input control, and then enter the corresponding branch in the branch input control, such as the test branch. Next, the developer clicks the "Optimize" control, triggering an update operation for the code analysis module. In response to this operation, the development device 102 executes a specific test corresponding to the test oracle. The specific test refers to using the aforementioned test oracle to test whether the expected fix code is effective. If the test passes, the expected fix code can be merged into the main branch and published to the code analysis tool, updating the code analysis module in the code analysis tool and enabling management of the code analysis tool. It should be noted that, considering reliability issues, the development device 102 can also perform regression testing on all correct test oracles. After the test passes, the operation of merging the expected fix code into the main branch is then executed.

[0098] In some possible implementations, development device 102 also supports code analysis tool developers in reviewing test oracles to ensure their accuracy and reasonableness. Code analysis tool developers can then process the code analysis module based on the reviewed and approved test oracles.

[0099] S222: Development device 102 sends processing results to service device 104.

[0100] S224: Service device 104 processes the code analysis module based on the processing results.

[0101] The development device 102 sends the processing result to the service device 104, so that the service device 104 can process the code analysis module according to the processing result. Specifically, the processing result includes at least one of regression test results or update results.

[0102] When the processing results include regression test results, if the regression test results indicate that the effectiveness of the code analysis module does not meet preset conditions, such as the false negative rate of the code analysis module reaching a false negative threshold, the service device 104 can also take the code analysis module offline. It will be re-launched when the developers of the code analysis tool optimize the module to ensure its effectiveness meets the preset conditions.

[0103] When the processing result includes an update result, the service device 104 updates the code analysis module according to the update result. In this way, the service device 104 can use the updated code analysis module to perform code analysis, which can reduce the false negative rate and / or false positive rate and improve the quality of code analysis.

[0104] S218 to S224 above describes how the service device 104 sends a test oracle to the development device 102, enabling the code analysis tool developer to process the code analysis module based on the test oracle, and then return the processing result to the service device 104 for illustrative purposes.

[0105] In some possible implementations, the service device 104 may also directly process the code analysis module based on test oracles. For example, the service device 104 may directly perform regression testing on the code analysis module using correct test oracles, and update the code analysis module using false negative or false positive test oracles to optimize false negatives and / or false positives.

[0106] It should also be noted that the user processing window of device 106 and the developer processing window of development device 102 can present their content in the form of web pages. Specifically, service device 104 can generate the source file of the user processing page based on defective code or system repair instances generated based on defective code, using web development technologies and tools, especially one or more dynamic web development technologies such as Hypertext Preprocessor (PHP), Java Server Pages (JSP), and Active Server Pages (ASP). User device 106 obtains the source file of the user processing page and generates a user processing window based on the source file, which includes the user processing page. In this way, the user of the code analysis tool can repair the defective code based on the user processing page.

[0107] Similarly, service device 104 can generate source files for a developer processing page using web development technologies and tools based on test predictions. Development device 102 obtains the source files of the developer processing page and generates a developer processing window based on these source files, including the developer processing page. Thus, the code analysis tool developer processes the code analysis module based on the developer processing page.

[0108] In some possible implementations, the user processing window of the application device 106 and the developer processing window of the development device 102 can also present their respective content as the user interface of the code analysis tool management system. Specifically, the development device 102, the service device 104, and the application device 106 constitute the code analysis tool management system, and the user interface of the code analysis tool management system can be stored in the development device 102 and the application device 106.

[0109] Specifically, the development device 102 can store the code analysis module processing user interface. The development device 102 can acquire test predictions and generate a developer processing window based on the test predictions and the code analysis module processing user interface. The developer of the code analysis tool processes the code analysis module through the code analysis module processing user interface in the developer processing window.

[0110] The device 106 can store defect code repair user interfaces, acquire defect codes or system repair instances, and generate a user processing window based on the defect code and the defect code repair user interface, or based on the system repair instance and the defect code repair user interface. Users of the code analysis module repair defect codes through the defect code repair user interface in the user processing window.

[0111] It should be noted that the implementation method of generating the processing window through a webpage makes this approach applicable to various platforms, such as Windows, Android, and iOS, exhibiting good compatibility. Generating the processing window through a user interface reduces the impact of network speed limitations, improves the smoothness of the code analysis tool's management system, and thus enhances the user experience.

[0112] The above text combines Figures 1 to 8 The present application describes in detail the management method of the code analysis tool provided in this application. The apparatus and device provided in this application will be described below with reference to the accompanying drawings.

[0113] See Figure 9 The schematic diagram of the service device 104 shown includes a communication unit 1042, a generation unit 1044, a management unit 1046, and an analysis unit 1040. The analysis unit 1040 includes a code analysis tool, which includes a code analysis module. The communication unit 1042 is used to obtain at least one user repair instance based on the user's repair behavior of the defective code in the code analysis tool. The generation unit 1044 is used to generate a test prediction based on at least one system repair instance generated by the analysis unit 1040 using the code analysis tool to repair the defective code based on the user repair instance; The management unit 1046 is also used to process the code analysis module according to the test prediction in order to manage the code analysis tool.

[0114] For a detailed implementation of the analysis unit 1040, please refer to [link to relevant documentation]. Figure 2 The embodiment shown describes the relevant content of S208. For a detailed implementation of the communication unit 1042, please refer to... Figure 2The embodiment shown describes the content related to S214. For a detailed implementation of the generation unit 1044, please refer to [link to relevant documentation]. Figure 2 The embodiment shown describes the relevant content of S216. For a detailed implementation of the management unit 1046, please refer to... Figure 2 Description of S224 in the illustrated embodiment.

[0115] In some possible implementations, the generation unit 1044 is specifically used for: Based on the association between the user repair instance and the system repair instance, at least one of the following is determined: a correct repair instance, a missed repair instance, and a false alarm repair instance; A test oracle is generated based on at least one of the correct repair instance, the missed repair instance, and the false positive repair instance, wherein the test oracle includes at least one of the correct test oracle, the missed repair instance, and the false positive test oracle.

[0116] In some possible implementations, the test oracle includes a correct test oracle; The management unit 1046 is specifically used for: Regression testing is performed on the code analysis module based on the correct test predictions to determine the effectiveness of the code analysis module.

[0117] In some possible implementations, the test oracle includes false negative test oracle and / or false positive test oracle; The management unit 1046 is specifically used for: The code analysis module is updated based on the false negative test prediction and / or false positive test prediction.

[0118] In some possible implementations, the user's code repair behavior includes the user's secondary repair behavior on the code after it has been repaired by the code analysis tool; When the generation unit 1044 determines at least one of the correct repair instance, the missed repair instance, and the false alarm repair instance based on the association between the user repair instance and the system repair instance, it is specifically used for: When the system repair instance fails to associate with the user repair instance, the system repair instance is determined to be the correct repair instance. When the system repair instance is successfully associated with the user repair instance, the system repair instance is determined to be a false alarm repair instance; When the association between the user repair instance and the system repair instance fails, the user repair instance is determined to be a missed repair instance.

[0119] In some possible implementations, the test oracle records an identifier for the defect code, which is used to index the defect code and its context.

[0120] According to the embodiments of this application, the service device 104 can correspondingly execute the methods described in the embodiments of this application, and the above and other operations and / or functions of each unit of the service device 104 are respectively for implementing Figure 2 For the sake of brevity, the corresponding processes of each method in the code will not be elaborated here.

[0121] The aforementioned service device 104 can be implemented via a computer cluster. Figure 10 A computer cluster is provided. Figure 10 The computer cluster shown is illustrated using a single computer as an example. Figure 10 As shown, the computer cluster 300 can be specifically used to implement the above. Figure 2 The illustrated embodiment describes the function of the service device 104. The computer cluster 300 includes a bus 301, a processor 302, a communication interface 303, and a memory 304. The processor 302, memory 304, and communication interface 303 communicate via the bus 301. The bus 301 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 10 The symbol is represented by a single thick line, but this does not indicate that there is only one bus or one type of bus. The communication interface 303 is used for communication with external devices. For example, the communication interface 303 can receive user actions to fix defective code from the code analysis tool sent by the device 106, or receive user fix instances sent by the device 106, etc.

[0122] The processor 302 may be a central processing unit (CPU). The memory 304 may include volatile memory, such as random access memory (RAM). The memory 304 may also include non-volatile memory, such as read-only memory (ROM), flash memory, HDD, or SSD.

[0123] The memory 304 stores executable code, and the processor 302 executes the executable code to perform the management method of the aforementioned code analysis tool.

[0124] Specifically, in achieving Figure 9 In the case of the illustrated embodiment, and Figure 9 When the units described in the embodiments are implemented in software, they perform... Figure 9 The software or program code required for the functions of the analysis unit 1040, generation unit 1044 and management unit 1046 are stored in the memory 304. The function of the communication unit 1042 is implemented through the communication interface 303. The processor is used to execute the instructions in the memory 304 and execute the management method of the code analysis tool applied to the service device 104.

[0125] Specifically, the computer cluster 300 receives user repair instances sent by the user device 106 through the communication interface 303. The processor 302 obtains user repair instances determined based on the user's repair behavior of the defective code by the code analysis tool from the communication interface through the bus 301, and then performs the following operations based on the instructions in the memory 304: At least one system repair instance generated based on the user repair instance and the code analysis tool's repair of the defective code generates a test prediction; The code analysis module is processed according to the test prediction to achieve management of the code analysis tool.

[0126] Optionally, the processor 302 is specifically configured to perform the following operations: Based on the association between the user repair instance and the system repair instance, at least one of the following is determined: a correct repair instance, a missed repair instance, and a false alarm repair instance; A test oracle is generated based on at least one of the correct repair instance, the missed repair instance, and the false positive repair instance, wherein the test oracle includes at least one of the correct test oracle, the missed repair instance, and the false positive test oracle.

[0127] Optionally, the test oracle includes a correct test oracle; The processor 302 is specifically used to perform the following operations: Regression testing is performed on the code analysis module based on the correct test predictions to determine the effectiveness of the code analysis module.

[0128] Optionally, the test oracle includes false negative test oracle and / or false positive test oracle; The processor 302 is specifically used to perform the following operations: The code analysis module is updated based on the false negative test prediction and / or false positive test prediction.

[0129] Optionally, the user's code repair behavior includes the user's secondary repair behavior on the code after it has been repaired by the code analysis tool; The processor 302 is specifically used to perform the following operations: When the system repair instance fails to associate with the user repair instance, the system repair instance is determined to be the correct repair instance. When the system repair instance is successfully associated with the user repair instance, the system repair instance is determined to be a false alarm repair instance; When the association between the user repair instance and the system repair instance fails, the user repair instance is determined to be a missed repair instance.

[0130] Optionally, the test oracle records an identifier for the defect code, which is used to index the defect code and its context.

[0131] It should be noted that, Figure 10 The example described uses a computer cluster 300, including one computer, to implement the function of the service device 104. In other possible implementations of this application, the function of the service device 104 may also be implemented by a computer cluster consisting of multiple computers.

[0132] In some possible implementations, the computer cluster may include multiple computers, each capable of performing all the functions of the service unit 104. That is, the memory 304 of each computer in the computer cluster stores the software or program code required to implement the functions of the analysis unit 1040, the generation unit 1044, and the management unit 1046.

[0133] In other possible implementations, a computer cluster may include multiple computers, with at least one computer implementing some of the functions of service device 104, and multiple computers able to cooperate to implement all the functions of service device 104. In one example, the computers in the computer cluster may be divided into several groups: one group of computers stores the software or program code required to implement the functions of analysis unit 1040 in its memory; another group of computers stores the software or program code required to implement the functions of generation unit 1044 in its memory; and yet another group of computers stores the software or program code required to implement the functions of management unit 1046 in its memory. In this way, all the functions of service device 104 are implemented through multiple computers.

[0134] It should be noted that the computers in the computer cluster can also be combined in other ways to implement all the functions of the service unit 104. For example, one group of computers can implement the functions of the analysis unit 1040, and another group of computers can implement the functions of the generation unit 1044 and the management unit 1046. This application does not limit this.

[0135] This application also provides a computer-readable storage medium including instructions that, when run on a computer cluster, cause the computer cluster to execute the management method of the code analysis tool applied to the service device 104 described above.

[0136] This application also provides a computer program product. When the computer program product is executed by a computer cluster, the computer cluster executes any of the aforementioned code analysis tool management methods. The computer program product can be a software installation package. When any of the aforementioned code analysis tool management methods is required, the computer program product can be downloaded and executed on the computer cluster.

[0137] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0138] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0139] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0140] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A method for managing code analysis tools, characterized in that, The code analysis tool includes a code analysis module, and the method includes: Based on the user's behavior in fixing defective code using the code analysis tool, obtain at least one user fix instance; Based on the association between the user repair instance and at least one system repair instance generated by the code analysis tool to repair the defective code, at least one of the following is determined: a correct repair instance, a missed repair instance, and a false alarm repair instance. A test prediction is generated based on at least one of the correct repair instance, the missed report repair instance, and the false positive repair instance, wherein the test prediction includes at least one of the correct test prediction, the missed report test prediction, and the false positive test prediction. The code analysis module is processed according to the test prediction to achieve management of the code analysis tool.

2. The method according to claim 1, characterized in that, The test oracle includes correct test oracles; The processing of the code analysis module based on the test prediction includes: Regression testing is performed on the code analysis module based on the correct test predictions to determine the effectiveness of the code analysis module.

3. The method according to claim 1, characterized in that, The test predictions include false negative test predictions and / or false positive test predictions; The processing of the code analysis module based on the test prediction includes: The code analysis module is updated based on the false negative test prediction and / or false positive test prediction.

4. The method according to any one of claims 1 to 3, characterized in that, The user's code repair behavior includes the user's secondary repair behavior on the code after it has been repaired by the code analysis tool; The step of determining at least one of correct repair instances, missed repair instances, and false positive repair instances based on the association between the user repair instance and at least one system repair instance generated by the code analysis tool in repairing the defective code includes: When the system repair instance generated by the code analysis tool after repairing the defective code fails to be associated with the user repair instance, the system repair instance is determined to be the correct repair instance. When the system repair instance generated by the code analysis tool after repairing the defective code is successfully associated with the user repair instance, the system repair instance is determined to be a false alarm repair instance. When the user repair instance fails to be associated with at least one system repair instance generated by the code analysis tool to repair the defective code, the user repair instance is determined to be a missed repair instance.

5. The method according to any one of claims 1 to 3, characterized in that, The test oracle records the identifier of the defect code, which is used to index the defect code and its context.

6. A service device, characterized in that, The device includes: a communication unit, a generation unit, a management unit, and an analysis unit; wherein the analysis unit includes a code analysis tool, and the code analysis tool includes a code analysis module; The communication unit is used to obtain at least one user repair instance based on the user's repair behavior of the defective code in the code analysis tool. The generation unit is configured to determine at least one of a correct repair instance, a missed repair instance, and a false alarm repair instance based on the association between the user repair instance and at least one system repair instance generated by the analysis unit using the code analysis tool to repair the defective code; and to generate a test prediction based on at least one of the correct repair instance, the missed repair instance, and the false alarm repair instance, wherein the test prediction includes at least one of a correct test prediction, a missed test prediction, and a false alarm test prediction. The management unit is also used to process the code analysis module according to the test prediction in order to manage the code analysis tool.

7. The apparatus according to claim 6, characterized in that, The test oracle includes correct test oracles; The management unit is specifically used for: Regression testing is performed on the code analysis module based on the correct test predictions to determine the effectiveness of the code analysis module.

8. The apparatus according to claim 6, characterized in that, The test predictions include false negative test predictions and / or false positive test predictions; The management unit is specifically used for: The code analysis module is updated based on the false negative test prediction and / or false positive test prediction.

9. The apparatus according to any one of claims 6 to 8, characterized in that, The user's code repair behavior includes the user's secondary repair behavior on the code after it has been repaired by the code analysis tool; The generation unit is specifically used for: When the system repair instance generated by the code analysis tool after repairing the defective code fails to be associated with the user repair instance, the system repair instance is determined to be the correct repair instance. When the system repair instance generated by the code analysis tool after repairing the defective code is successfully associated with the user repair instance, the system repair instance is determined to be a false alarm repair instance. When the user repair instance fails to be associated with at least one system repair instance generated by the code analysis tool to repair the defective code, the user repair instance is determined to be a missed repair instance.

10. The apparatus according to any one of claims 6 to 8, characterized in that, The test oracle records the identifier of the defect code, which is used to index the defect code and its context.

11. A computer cluster, characterized in that, It includes at least one computer, each computer including a processor and memory; The processor of the at least one computer is configured to execute instructions stored in the memory of the at least one computer to perform the method as described in any one of claims 1 to 5.

12. A computer-readable storage medium, characterized in that, Includes instructions that, when run on a computer cluster, cause the computer cluster to perform the method as described in any one of claims 1 to 5.

Images