Information processing apparatus and information processing method, computer program, and information processing system

Abstract

Provided is an information processing device that performs a process related to code analysis of software in an intermediate state.　The information processing device is provided with: a differential information acquisition unit that acquires differential information between code saved in a first repository, and code corresponding to said code saved in a second repository; and a code analysis unit that reproduces the code saved in the first repository from the code saved in the second repository on the basis of the differential information, and performs code analysis on the reproduced code. The information processing device reports the analysis result by the code analysis unit to the developer of the code.

Description

Technical Field

[0001] The technology disclosed in this specification (hereinafter referred to as "this disclosure") relates to an information processing apparatus and method for performing processing related to code analysis of software, a computer program, and an information processing system.

Background Art

[0002] Recently, with the development of ICT (Information and Communication Technology), the needs for software have been increasing more and more, and a large amount of software is being developed every day. Note that software and computer programs as referred to in this specification are almost synonymous. Unless otherwise specified, software and computer programs are assumed to be in the state of source code as developed by the developer.

[0003] It is common to perform code analysis on the developed software. In code analysis, the control flow, structure, class inheritance of the program, the relationship between functions and variables are analyzed, or the vulnerability of functions is verified (for example, refer to Patent Document 1). The methods of code analysis can be roughly classified into two types: a method performed on a local PC (Personal Computer) used by software developers and a method performed in a cloud environment.

[0004] According to the former method of performing code analysis on a local PC, every time a commit (updating the content of the edited and updated file) is made in the local environment, code analysis can be performed in a short time and notified to the developer at an early stage. Recently, although an analysis function equivalent to that of the cloud is provided on the local PC, there are problems such as many function limitations and limited computing resources. On the other hand, according to the latter method of performing code analysis in a cloud environment, the computing speed can be improved by utilizing rich computing resources and parallel processing.

[0005] For example, CircleCI provides a cloud-based code analysis service, while also offering tools for code analysis on local PCs. Similarly, GitHub offers "GitHub Actions," a service that performs CI and code analysis in the cloud. Furthermore, a third-party developer provides "Act," a tool for running "GitHub Actions" in a local environment. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2019-168753 [Non-patent literature]

[0007] [Non-Patent Document 1] Mateusz Machalica,Alex Samylkin,Meredith Porth,Satish Chandra,"Predictive Test Selection" [Non-Patent Document 2] Claire Leong,Abhayendra Singh,Mike Papadakis,Yves Le Traon,John Micco,"Assessing Transition-based Test Selection Algorithms at Google" [Overview of the project] [Problems that the invention aims to solve]

[0008] The purpose of this disclosure is to provide an information processing device and information processing method, a computer program, and an information processing system that perform code analysis of software in an intermediate state before it is uploaded to a cloud environment. [Means for solving the problem]

[0009] This disclosure has been made in consideration of the above issues, and its first aspect is: A difference information acquisition unit that acquires difference information between the code stored in the first repository and the code corresponding to the said code stored in the second repository, A code analysis unit that, based on the difference information, reproduces the code stored in the first repository from the code stored in the second repository and performs code analysis on the reproduced code, It is an information processing device equipped with the following features.

[0010] The difference information acquisition unit acquires the difference information based on a list of commits that have not yet been pushed to the second repository. Specifically, if there is a branch in the code history, the difference information acquisition unit acquires the difference information based on a list of commits necessary to reconstruct the latest code stored in the first repository from the branch.

[0011] The code analysis unit then reproduces the latest code stored in the first repository based on the information of each commit included in the list of commits, and performs code analysis on the reproduced code.

[0012] Furthermore, the code analysis unit performs tests using test codes selected from a plurality of test codes defined for the code. In selecting the tests, the code analysis unit uses a machine learning model that has learned from the history of tests using modified files and test codes, and also incorporates rule-based selection.

[0013] Furthermore, the second aspect of this disclosure is, A difference information acquisition step that acquires difference information between the code stored in the first repository and the code corresponding to the said code stored in the second repository, A code analysis step which involves reconstructing the code stored in the first repository from the code stored in the second repository based on the difference information, and performing code analysis on the reconstructed code, This is an information processing method that possesses [certain properties].

[0014] Furthermore, the third aspect of this disclosure is, A difference information acquisition unit that acquires difference information between the code stored in the first repository and the code corresponding to the said code stored in the second repository. A code analysis unit that, based on the difference information, reproduces the code stored in the first repository from the code stored in the second repository, and performs code analysis on the reproduced code. It is a computer program written in a computer-readable format to make a computer function.

[0015] The computer program relating to the third aspect of this disclosure defines a computer program written in a computer-readable format to perform a predetermined process on a computer. In other words, by installing the computer program relating to the third aspect of this disclosure on a computer, collaborative effects can be achieved on the computer, and the same effects as those of the information processing device relating to the first aspect of this disclosure can be obtained.

[0016] Furthermore, the fourth aspect of this disclosure is, A commit information acquisition device that acquires commit information regarding commits that represent the difference between the latest code developed in the local environment and the code corresponding to said code that has been pushed to the remote repository, A code analysis device that, based on the commit information acquired by the commit information acquisition device, reproduces the latest code from the code already pushed to the remote repository and performs code analysis on the reproduced code, It is an information processing system equipped with the following features.

[0017] However, the "system" mentioned here refers to a logical aggregation of multiple devices (or functional modules that realize specific functions), and it does not particularly matter whether each device or functional module is in a single housing.

Advantages of the Invention

[0018] According to the present disclosure, it is possible to provide an information processing apparatus, an information processing method, a computer program, and an information processing system for performing processing related to code analysis of software in commit units.

[0019] Note that the effects described in this specification are merely examples, and the effects brought about by the present disclosure are not limited thereto. Also, the present disclosure may have additional effects other than the above effects.

[0020] Further other objects, features, and advantages of the present disclosure will become apparent from more detailed descriptions based on the embodiments described later and the attached drawings.

Brief Description of the Drawings

[0021] [Figure 1] FIG. 1 is a diagram showing a software development environment 100 assuming code analysis services in a cloud environment. [Figure 2] FIG. 2 is a diagram showing a procedure for performing code analysis on source code in an intermediate state of development in the software development environment 100. [Figure 3] FIG. 3 is a flowchart showing a procedure for performing code analysis on source code in an intermediate state of development. [Figure 4] FIG. 4 is a diagram showing a functional configuration example for performing code testing. [Figure 5] FIG. 5 is a diagram showing an example of performing test selection based on a machine learning model. <0°000111>FIG. 6 is a diagram showing an internal configuration example of the test selection unit 403. [Figure 7]Figure 7 shows an example of the internal configuration of the test selection unit 403, which applies rules according to the conditions. [Figure 8] Figure 8 illustrates a specific example of selecting a test target file based on the probability of failure. [Figure 9] Figure 9 shows an example of the functional configuration of the test selection unit 403 during the learning phase for generating a machine learning model. [Figure 10] Figure 10 shows an example of the functional configuration of the test selection unit 403 for performing test selection in the prediction phase. [Figure 11] Figure 11 shows an example of how to select a test by applying rules in the test selection unit 403 shown in Figure 10. [Figure 12] Figure 12 shows another application example of applying rules to select tests in the test selection unit 403 shown in Figure 10. [Figure 13] Figure 13 shows an example of a configuration file that includes a description of the rule definition. [Figure 14] Figure 14 shows an example of a software development project flow including branches. [Modes for carrying out the invention]

[0022] The present disclosure will be described below in the following order, with reference to the drawings.

[0023] A. Overview B. Code analysis in an intermediate state C. Selecting Test Code D. Specific Examples Regarding Test Code Selection

[0024] A. Overview In software development, code analysis is preferably performed in a cloud environment with abundant computing resources and capabilities. Figure 1 schematically shows a software development environment 100 that assumes the provision of code analysis services in a cloud environment. The software development environment 100 is broadly divided into a local environment 110 and a cloud environment 120. For the sake of simplification, only one local environment 110 and one cloud environment 120 are depicted in Figure 1, but in reality, it is assumed that one cloud environment 120 provides code analysis services to numerous local environments that are not shown.

[0025] The local environment 110 corresponds to the software development site and includes a local PC 111 used by developers for software development tasks such as creating and modifying source code, and a local repository 112 where the source code is stored.

[0026] On the other hand, the cloud environment 120 includes a remote repository 121, which is the location for storing source code on the cloud side, and a code analysis service unit 122 that performs code analysis on the source code in the remote repository. Code analysis includes analysis of program control flow and structure, class inheritance, relationships between functions and variables, and vulnerability verification of functions, but the analysis functions provided by the code analysis service unit 122 are not limited to these, and it is possible to add or modify functions to the code.

[0027] Developers use their local PC 111 to create, edit, and update source code (such as adding features or fixing bugs), and to change versions as the source code is repeatedly updated. Saving source code files edited and updated on the local PC 111 to the local repository 112 is called a "commit." Uploading source code from the local environment 110 to the remote repository 121 is also called a "push." ​​Conversely, downloading source code from the remote repository 121 is also called a "pull."

[0028] Furthermore, in order to maintain the consistency of the software under development in the software development environment 100, a version control system (VCS) 113 is installed in the local environment 110. The version control system 113 issues a commit ID each time a commit is made to the local repository 112, and associates a description of the commit's contents, such as feature additions, bug fixes, or version maintenance, with the commit ID. When source code is committed to the local repository 112, the commit history information associated with the commit ID is recorded along with the source code. Also, when source code is uploaded to the remote repository 121, the source code is recorded in the remote repository 121, associated with the commit ID.

[0029] Furthermore, a commit branching function has been incorporated to enable parallel software development of multiple versions. In this case, the commit history will have a tree structure, and the local repository 112 will manage the commit history in a tree structure format.

[0030] Figure 14 illustrates the workflow of a software development project including branches. The horizontal axis in the figure represents time, progressing from left to right. The main project branch is master branch 1401. On master branch 1401, the version of the software (source code) under development is repeatedly changed. Each version change involves a commit to the local repository 112 and an upload (push) to the remote repository 121. It is expected that one or more commits will be made within a version, but a push is not performed for every commit. Branches 1402 and 1403 are spun off from master branch 1401 for purposes such as adding features to the source code or fixing bugs. The contents of the branched branches do not affect other branches, allowing multiple people to work on them simultaneously. Furthermore, by keeping a history for each branch, it becomes easier to isolate and address problems such as bugs when they occur. The development history, including commit information (developer, message, patch, etc.), is recorded in each repository. Although not shown in Figure 14, the branched branches 1402 and 1403 may be merged into the master branch 1401 in the future.

[0031] Here, we will re-examine in detail the problems that arise when performing code analysis in a cloud environment 120.

[0032] When performing code analysis in the cloud environment 120, it is assumed that the source code files to be analyzed have been uploaded to the remote repository 121. Here, the source code uploaded to the remote repository 121 must be the final source code, including test code; it is not possible to analyze source code in an "intermediate state" during development. In other words, it is not possible to analyze "intermediate state" source code that has only been committed to the local repository 112 in the cloud environment 120. Therefore, it is not possible to perform code analysis quickly and notify developers at an early stage.

[0033] On the other hand, performing code analysis on source code in an intermediate development state requires doing so in the local environment 110, which presents problems in terms of limitations in computing resources and functionality.

[0034] Furthermore, when performing source code analysis during the development process, it is necessary to appropriately and efficiently select the required test code according to the modified parts of the source code and ensure that sufficient tests are performed.

[0035] B. Code analysis in an intermediate state Figure 2 shows the procedure for performing code analysis on source code in an intermediate development state, which has not yet been uploaded to the remote repository 121, within the software development environment 100. Below, the code analysis procedure will be explained using the example of a case where the code analysis process is started each time the software developer commits source code.

[0036] The software developer commits the source code they edited and updated on their local PC 111. In response, the version control system 113 issues a new commit ID and records a description of the commit's contents, associating it with the commit ID. The source code edited and updated on the local PC 111 is then saved to the local repository 112, associated with the issued commit ID.

[0037] The version control system 113 generates a commit event when it issues a new commit ID. The commit information acquisition unit 114 responds to this commit event and checks the latest commit ID of the source code to be analyzed in the local environment 110.

[0038] Next, the commit information acquisition unit 114 checks the latest commit ID among the source code stored in the remote repository 121 that is historically related to the source code to be analyzed.

[0039] Next, the commit information acquisition unit 114 refers to the commit history information managed in the version control system 113 to search for the branch (fork point) between the latest commit ID on the local environment 110 and the latest commit ID on the remote environment 120, and creates a list of commits necessary to reconstruct the latest commit ID on the local environment 110 from the commit ID of that branch. This list of commits is equivalent to a list of commits that have not yet been pushed to the remote repository 121.

[0040] Then, for each commit in the created list of commits, the commit information acquisition unit 114 acquires commit information from the local environment 110 (specifically, the local repository 112, etc.), consisting of information such as the developer, messages related to the commit, and patches, and sends it to the code analysis service unit 122 on the cloud environment 120.

[0041] In response, the code analysis service unit 122 retrieves (downloads) the latest code from the remote repository 121 that corresponds to the source code to be analyzed, and then, based on the commit information received from the commit information retrieval unit 114, reproduces the latest source code in the intermediate state to be analyzed.

[0042] The code analysis service unit 122 then performs code analysis on the reproduced source code and sends the analysis result report to the local PC 111.

[0043] Therefore, by following the procedure shown in Figure 2, it is possible to perform code analysis on "intermediate state" source code that has only been committed to the local repository 112, and notify the developer at an early stage.

[0044] In Figures 1 and 2, the version control system 113 consists of a distributed version control system that records and tracks the change history of source code, such as Git.

[0045] Furthermore, the commit information acquisition unit 114 consists of a script, such as a shell script, that is called when the version control system 113 creates a commit, and sends the information necessary to reproduce the source code to the cloud (code analysis service unit 122).

[0046] Furthermore, the code analysis service unit 122 is a service that performs actual code analysis, such as Rocro's Inspecode. In the software development environment 100 shown in Figure 2, the code analysis service unit 122 reproduces code equivalent to that in the local environment.

[0047] Furthermore, the remote repository 121 corresponds to a git server located in the cloud environment 120, or a web service with VCS functionality (such as GitHub).

[0048] Figure 3 shows a flowchart illustrating the procedure for performing code analysis on source code in an intermediate development state within the software development environment 100.

[0049] When a commit is made to the source code edited and updated on the local PC 111 (Yes in step S301), the version control system 113 issues a new commit ID. The version control system 113 also records a description of the commit contents, associating it with the issued commit ID.

[0050] The commit information acquisition unit 114, in response to a commit event when a new commit ID is issued, checks the version control system 113 for the latest commit ID of the source code to be analyzed in the local environment 110 (step S302).

[0051] Next, the commit information acquisition unit 114 checks the latest commit ID among the source code stored in the remote repository 121 that is historically related to the source code to be analyzed (step S303).

[0052] Next, the commit information acquisition unit 114 refers to the code history information managed in the local repository 112 or the version control system 113 to search for the branch (fork point) between the latest commit ID on the local environment 110 and the latest commit ID on the remote environment 120 (step S304).

[0053] Next, the commit information acquisition unit 114 creates a list of commits necessary to reproduce the latest commit ID on the local environment 110 from the commit ID of the found branch, that is, a list of commits that have not yet been pushed to the remote repository 121 (step S305).

[0054] Then, the commit information acquisition unit 114 acquires commit information for each commit in the created commit list and sends it to the code analysis service unit 122 on the cloud environment 120 (step S306).

[0055] The code analysis service unit 122 reproduces the contents of the latest commit on the local environment 110, i.e., the latest source code in an intermediate state, based on the commit information received from the commit information acquisition unit 114 (step S307).

[0056] Then, the code analysis service unit 122 performs code analysis on the reproduced source code (step S308). After that, the code analysis service unit 122 sends an analysis result report regarding the latest reproduced source code to the local PC 111 (step S309) and terminates this process.

[0057] In this way, the local environment 110 can identify the difference information between the latest commit on the local environment 110 and the latest commit on the remote environment 120 based on the commit history information managed by, for example, the version control system 113, and provide it to the remote environment 120. Then, on the remote environment 120 side, the code analysis service unit 122 can reproduce the source code based on the difference information provided from the local environment 110, thereby realizing a code analysis service for intermediate source code that has not yet been uploaded to the remote repository 121.

[0058] C. Selecting Test Code According to the code analysis method described in section B above, it becomes possible to perform tests on source code in an intermediate development state before uploading the source code to the remote repository 121. Therefore, there is a further need to reduce the execution time of frequently performed tests.

[0059] If the code analysis service unit 122 were to run all predefined test codes for the source code being analyzed every time a code analysis is performed (for example, every time a code analysis is performed after each commit), the test execution time would become very long. Therefore, it is desirable to adaptively select and run some of the test codes rather than running all of them every time.

[0060] Figure 4 schematically shows an example of the functional configuration for performing code testing in the code analysis service unit 122. Each functional block will be described below.

[0061] The code acquisition unit 401 acquires the source code that is the target of code analysis (or test execution). The source code referred to here is assumed to be the latest source code reproduced by the code analysis service unit 122 according to the processing flow shown in Figure 2.

[0062] The test information acquisition unit 402 acquires information on all test codes that are predefined for the source code to be tested (for example, test target file names). For example, if the file names of the test codes to be applied to the source code are written within the source code, the test information acquisition unit 402 acquires the test code information from within the source code. Of course, the information on the test codes defined for each source code may also be stored in a location other than the source code.

[0063] The test selection unit 403 selects the test code to be executed from all the predefined test codes for the source code, which have been acquired by the test information acquisition unit 402. Then, the test execution unit 404 performs tests on the source code using the test code selected by the test selection unit 404. The test results from the test execution unit 404, along with other code analysis results from the code analysis service unit 122, are reported to the developer or other relevant parties.

[0064] As mentioned above, it is desirable for the test selection unit 403 to adaptively select and execute some test codes rather than executing all test codes every time.

[0065] For example, a method has been proposed for selecting test codes using machine learning models (see Non-Patent Documents 1 and 2).

[0066] Non-Patent Document 1 proposes a method for creating a machine learning model by extracting features from information about the modified source code (changed file) and test code, and using this machine learning model to select test code that is likely to fail on the changed file. Non-Patent Document 1 describes a source code management method in which only source code that passes tests is registered in the repository, and modifications that fail tests are not incorporated into the source code. In other words, in Non-Patent Document 1, code that fails tests is not included in the repository, but it is thought that the test execution results are stored separately for the generation of datasets used in machine learning. Therefore, it is confirmed that file changes due to commits are the cause of test failures (if source code that is already in a state where tests fail exists in the repository, it is possible that the test fails not only due to file changes due to commits, but also due to older commits). In other words, in Non-Patent Document 1, there is a risk that previously failed test code will be less likely to be run on the changed file. Non-Patent Document 1 also describes that all tests without test selection are run periodically.

[0067] Furthermore, Non-Patent Document 2, like Non-Patent Document 1, describes using a machine learning model to select test code for source code. In the method described in Non-Patent Document 2, source code tests are performed in batches of file changes, resulting in situations where the test results are unknown. Non-Patent Document 2 describes creating a model for each transition in the test results, such as from success to failure, success to not performed, and not performed to failure, to evaluate the test selection.

[0068] One possible approach is to create a machine learning model that uses the modified files created by the developer and the test history (whether the test code succeeded or failed) as training data to select the appropriate test code from the modified files. However, this model cannot be used when the source code (file) is modified after a test fails and then a test is selected. Therefore, it is necessary to create a machine learning model using the modified files and test history created by the developer as training data when the previous test failed, and then create yet another machine learning model using the modified files and test history created by the developer as training data when the previous test succeeded (see Figure 5), which is cumbersome.

[0069] Therefore, this disclosure proposes a simplified test selection method that combines prediction using machine learning models with rule-based selection.

[0070] Figure 6 shows an example of the internal configuration of the test selection unit 403, which is configured based on this disclosure, as shown in the functional block diagram in Figure 4.

[0071] The machine learning model 601 is pre-trained on the modified files after source code modifications and the history of past tests. The trained machine learning model 601 predicts one or more appropriate test codes for the modified files after source code modifications.

[0072] The evaluation unit 602 evaluates one or more test codes predicted by the machine learning model 601 based on predetermined rules, outputs one or more test codes to be performed for the current modified file, and uses this as the final test selection result.

[0073] As shown in Figure 6, the configuration of the test selection unit 403 allows for rule-based selection rather than relying solely on predictions from the machine learning model 601. Therefore, for example, by applying a rule in the evaluation unit 602 to prioritize the execution of tests that failed in the previous test, it is possible to achieve the operation of prioritizing the selection of tests that failed in the previous test for modified files that failed in the previous test.

[0074] As shown in Figure 6, only one machine learning model is needed for test selection, making management and calibration (retraining the model) easier.

[0075] In the evaluation unit 602, the following rules (1) to (3) may also be applied.

[0076] (1) Rules based on branch name For example, for the master branch repository, a rule can be established to perform a full test selection instead of test selection. In this way, it becomes unnecessary to perform full tests periodically as in Non-Patent Document 1, and full tests can be performed only on the necessary branches.

[0077] (2) Rules to adjust the selection rate of tests for continuous integration (CI) for a specific branch name. Continuous integration is a software development methodology that automates the software development process, automatically running builds and tests as developers commit source code and merge it into a shared repository. Applying this rule helps prevent false negatives in test selection.

[0078] (3) Rules for assigning priority to specific tests Prioritization can be assigned to specific tests by applying an offset to the prediction results of machine learning model 601. Whether or not a test is a specific one can be identified based on, for example, the string pattern of the test target file name.

[0079] If multiple rules are available, you can pre-set conditions for each rule and then apply the rule that matches the conditions when selecting a test.

[0080] Figure 7 shows an example of the functional configuration of the test selection unit 403, which is configured to select tests using rules that meet certain conditions. However, the same reference numbers are used for components that are the same as those shown in Figure 6.

[0081] The rule storage unit 702 stores multiple rules that are applied when selecting test code. Each rule has conditions set for its application. The rule reading unit 701 reads the rule that matches the conditions from the rule storage unit 702 when selecting a test.

[0082] The evaluation unit 602 then evaluates one or more test codes predicted by the machine learning model 601 based on the rules read by the rule reading unit 701, and outputs one or more test codes to be performed for the modified file, which becomes the final test selection result. The actions set in the rules include, for example, adjusting priority predictions and adjusting test selection rates. The evaluation unit 602 selects tests based on the selection rate and test failure prediction values ​​based on rules that meet the conditions.

[0083] Furthermore, if the rule reading unit 701 cannot read a rule that matches the conditions from the rule storage unit 702 (i.e., if no rule that matches the conditions exists), the evaluation unit 602 will output the final test selection result using the prediction results from the machine learning model 601 as is, without evaluating the test code predicted by the machine learning model 601 based on the rules.

[0084] D. Specific Examples Regarding Test Code Selection Section D describes specific examples of test code selection.

[0085] D-1. How to select a test It is desirable to selectively run only the tests that are likely to fail due to the modified files (source code), and to avoid unnecessarily running tests that are likely to succeed. Figure 8 shows an example of test selection based on the probability of failure. The upper part of the figure shows the results of calculating the probability that the file under analysis (source code) will fail each test code (test target file) using a predetermined algorithm. The lower part of the figure shows the result of selecting a subset of the defined test target files, consisting of two test target files with a high probability of failure, aaa / eee_test.go and aaa / fff / ggg_test.go.

[0086] D-2. About the Learning Phase In the training phase of machine learning model 601, the model is trained using features extracted from metadata related to the modified source code files and test target files, as well as the test target history (test results). Using this trained machine learning model, the probability of a test failing can be predicted for each test target file.

[0087] D-3. Feature Generation Examples of features used to train machine learning model 601 include (1) to (4) below.

[0088] (1) The number of changes made to the modified file in the past 14 days. (2) The probability of a test target file failing over the past 14 days. (3) The degree of match between the file paths of the modified file and the test target file. (4) A model is generated by training it using a list of feature IDs and their values ​​for the test target file, and the test results as training data.

[0089] D-4. Feature generation The number of changes to modified files over the past 14 days, and a list of files changed between two commits, can be obtained from the repository as follows:

[0090]

number

[0091] Furthermore, the commit creation date and time can be obtained as follows:

[0092]

number

[0093] Furthermore, the number of commits for each modified file over the past 14 days can be obtained as follows:

[0094]

number

[0095] Then, the feature ID and its value are connected with a colon, and a list is created by combining it with other features.

[0096] D-5. About Model Learning For each test target file, a list in LibSVM (Library for Support Vector Machines) format is generated, consisting of a single-line label, a feature ID, and its value. Note that LibSVM format is a feature vector file format required as training data by many machine learning libraries.

[0097]

number

[0098] In each entry of the list above, the leading number is the label, where 0 indicates a successful test and 1 indicates a failed test.

[0099] Based on the training data described above, a model is created using a Boosted Decision Tree. Then, this model is used to predict failures for each test target file.

[0100] D-6. About the Prediction Phase In the prediction phase, a list of features is generated from the files that were modified when the source code was pushed or pulled to the remote repository 121, and from the test target files. Using the trained model described above, a failure prediction for each test target file is obtained from the list of features.

[0101]

number

[0102] The trained model outputs a failure prediction score for each test target file, ranging from 0 to 1. The evaluation unit 602 selects a subset of test target files based on the test selection rate or score threshold.

[0103] D-7. Regarding the problems The trained machine learning model 601 can be used to predict the probability that a modified file will fail its tests. However, if the modified file does not correct the part of the source code that caused the previous test to fail, the machine learning model 601 cannot predict the probability that the previously failed test will fail again.

[0104] The technology described in Non-Patent Document 1 does not address this situation; in other words, it does not address this issue. Furthermore, the method described in Non-Patent Document 2 requires the creation of a separate learning model for each transition type, which is cumbersome as it necessitates collecting learning models for each model, and also makes management and calibration difficult.

[0105] In contrast, this disclosure allows for the application of a rule to offset the prediction results of the machine learning model 601, thereby setting a higher probability of failure for tests that previously failed.

[0106] D-8.Functional configuration Figure 9 shows an example of the functional configuration of the test selection unit 403 during the learning phase for generating a machine learning model.

[0107] When a dataset consisting of source code metadata and past test results (test history) is read from the repository as training data, the feature generation unit 901 generates a LibSVM format feature vector file from this dataset, which becomes input to the model generation unit 902. The model generation unit 902 consists of a Gradient Boosting Framework and uses the above dataset for training, generating a machine learning model 903 that predicts the probability of test failure from the source code metadata and past test results.

[0108] Figure 10 shows an example of the functional configuration of the test selection unit 403 for performing test selection in the prediction phase. In this prediction phase, the machine learning model generated in the learning phase shown in Figure 9 is used. Test selection is performed when the software source code is changed.

[0109] The feature generation unit 1001 generates a LibSVM format feature vector file, which will be used as input to the trained machine learning model, from the metadata of the modified source code and the dataset of past test results (test history).

[0110] The prediction score generation unit 1002 uses the machine learning model 903 generated in the learning phase shown in Figure 9 to generate a failure prediction score for when the modified source code (changed file) is tested, based on the input LibSVM format feature vector file.

[0111] Meanwhile, the comparison unit 1003 compares the status of the previous test (success or failure), the branch name of the source code being tested, and the test targets described in the source code, and outputs rules that match the conditions.

[0112] The operation unit 1004 manipulates the predicted failure score and selection rate of each test code generated by the predicted score generation unit 1002. Then, the final selection unit 1005 ultimately selects the test code to be executed for the target source code based on the results of the manipulation.

[0113] Figure 11 shows a specific application example of applying a rule to select a test in the test selection unit 403 shown in Figure 10. Here, the rule applied is to add an offset corresponding to the state of the previous test to the failure prediction score.

[0114] The feature generation unit 1001 generates a LibSVM-format feature vector file, which serves as input to the trained machine learning model, from the metadata of the modified source code and the dataset of past test results (test history). The prediction score generation unit 1002 then generates a failure prediction score for when the modified file is tested, based on the input LibSVM-format feature vector file. In this case, if the changes in the modified file are unrelated to the test, the failure prediction score is predicted to be 0.1.

[0115] Meanwhile, the comparison unit 1003 compares the above rule with the status of the previous test. Since the status of the previous test was Failed, the comparison unit 1003 outputs a rule that matches the condition and adds 0.8 to the failure prediction score of that test.

[0116] Therefore, the operation unit 1004 applies the rule of adding 0.8 to the failure prediction score of 0.1 for the previously failed test, making the failure prediction score for that test 0.9.

[0117] As a result, although the test target is not associated with the modified file, an offset is added because it failed in the previous test, resulting in a high failure prediction score of 0.9. Therefore, the final selection unit 1005 selects that test target.

[0118] Figure 12 shows another application example of applying rules to select tests in the test selection unit 403 shown in Figure 10. Here, the rule is applied to select all tests for the master branch repository.

[0119] The feature generation unit 1001 generates a LibSVM-format feature vector file, which will be used as input to the trained machine learning model, from the metadata of the modified source code and the dataset of past test results (test history). The prediction score generation unit 1002 then generates a failure prediction score for when the modified file is tested, from the input LibSVM-format feature vector file.

[0120] Meanwhile, the comparison unit 1003 compares the above rule with the state of the previous test. Here, the comparison unit 1003 checks the branch name of the input source code and, since it matches the condition as master, outputs a rule to select all tests, i.e., to set the selection rate to 100%.

[0121] Therefore, the operation unit 1004 sets the selection rate to 100% for the final selection unit 1005. As a result, the final selection unit 1005 selects all test targets regardless of the score value predicted by the prediction score generation unit 1002.

[0122] Furthermore, the rules to be applied in the test selection unit 403 may be made available to the user (such as a source code developer or administrator) through UI (User Interface) operations.

[0123] For example, as shown in Figure 13, you can add a description of the rules applicable to test selection to the configuration file in YAML (YAML Ain't Markup Language) format. In the example definition shown in Figure 13, each rule has a rule name, a condition for applying the rule, and an action to be applied. For example, at the start of code analysis, the user could be allowed to specify the rules they want to apply to test selection from a UI screen (not shown) consisting of a list of rule names. [Industrial applicability]

[0124] The present disclosure has been described in detail above with reference to specific embodiments. However, it will be obvious that those skilled in the art can modify or substitute these embodiments without departing from the gist of the present disclosure.

[0125] According to this disclosure, by using a code analysis service installed in the cloud, it is possible to perform code analysis on intermediate source code, for example, after each commit, and report the code analysis results to the developer at an early stage. Because a code analysis service on the cloud is used, code analysis can be performed at an early stage of the code development process without limitations on computing resources or functionality. Of course, it is also possible to implement a similar code analysis service in a local environment.

[0126] In short, this disclosure has been explained in the form of examples, and the contents of this specification should not be interpreted restrictively. The claims should be considered in order to determine the gist of this disclosure.

[0127] Furthermore, this disclosure may also take the following form.

[0128] (1) A difference information acquisition unit that acquires difference information between the code stored in the first repository and the code corresponding to the said code stored in the second repository, A code analysis unit that, based on the difference information, reproduces the code stored in the first repository from the code stored in the second repository and performs code analysis on the reproduced code, An information processing device equipped with the following.

[0129] (2) The code analysis results obtained by the code analysis unit are notified to the developer of the code. The information processing device described in (1) above.

[0130] (3) The difference information acquisition unit acquires the difference information based on a list of commits that have not yet been pushed to the second repository. The information processing device described in either (1) or (2) above.

[0131] (4) If there is a branch in the code history, the difference information acquisition unit acquires the difference information based on a list of commits necessary to reproduce the latest code stored in the first repository from the branch. The information processing device described in (3) above.

[0132] (5) The code analysis unit reproduces the latest code stored in the first repository based on the information of each commit included in the list of commits, and performs code analysis on the reproduced code. The information processing device described in (4) above.

[0133] (6) The code analysis unit performs a test using a test code selected from among a plurality of test codes defined for the code. An information processing device as described in any of (1) to (5) above.

[0134] (7) The code analysis unit selects test code using a machine learning model that has learned the history of tests using modified files and test code. The information processing device described in (6) above.

[0135] (8) The code analysis unit selects test code by applying rules to the prediction results from the machine learning model. The information processing device described in (7) above.

[0136] (9) The aforementioned rules include at least one of the following: (a) A rule to prioritize the execution of tests that failed last time. (b) A rule to run all tests according to the branch name. (c) A rule to adjust the test selection rate for continuous integrations to a specific branch name. (d) Rules for assigning priority to specific tests The information processing device described in (8) above.

[0137] (10) Each of the multiple rules has a condition set, The code analysis unit selects test code by applying rules that match the conditions to the prediction results. The information processing device described in either (8) or (9) above.

[0138] (11) A step of obtaining difference information by obtaining difference information between the code stored in the first repository and the code corresponding to the said code stored in the second repository, A code analysis step which involves reconstructing the code stored in the first repository from the code stored in the second repository based on the difference information, and performing code analysis on the reconstructed code, An information processing method having

[0139] (12) Difference information acquisition unit that acquires difference information between the code stored in the first repository and the code corresponding to the said code stored in the second repository. A code analysis unit that, based on the difference information, reproduces the code stored in the first repository from the code stored in the second repository, and performs code analysis on the reproduced code. A computer program written in a computer-readable format to enable a computer to function.

[0140] (13) A commit information acquisition device that acquires commit information relating to the commit that is the difference between the latest code developed in the local environment and the code corresponding to the said code that has been pushed to the remote repository, A code analysis device that, based on the commit information acquired by the commit information acquisition device, reproduces the latest code from the code already pushed to the remote repository and performs code analysis on the reproduced code, An information processing system equipped with the following features. [Explanation of Symbols]

[0141] 100…Software development environment, 110…Local environment 111...Local PC, 112...Local repository 113... Version control system, 114... Commit information acquisition unit 120...Cloud environment, 121...Remote repository 123... Code Analysis Service Department 401...Code acquisition unit, 402...Test information acquisition unit 403...Test Selection Unit, 404...Test Execution Unit 601…Machine Learning Models, 602…Evaluation Department 701 Rule reading unit, 702... Rule storage unit 901...Feature generation unit, 902...Model generation unit 903... Machine learning model 1001...Feature generation unit, 1002...Prediction score generation unit 1003...Comparison section, 1004...Operation section, 1005...Final selection section

Claims

1. A difference information acquisition unit obtains difference information between the code stored in the first repository and the corresponding code stored in the second repository, based on a list of commits that have not yet been pushed to the second repository. A code analysis unit that reproduces the code stored in the first repository from the code stored in the second repository based on the difference information, and performs code analysis on the reproduced code, An information processing device equipped with the following.

2. The code analysis results obtained by the code analysis unit are notified to the developer of the code. The information processing apparatus according to claim 1.

3. If there is a branch in the code history, the difference information acquisition unit acquires the difference information based on a list of commits necessary to reproduce the latest code stored in the first repository from the branch. The information processing apparatus according to claim 1.

4. The code analysis unit reproduces the latest code stored in the first repository based on the information of each commit included in the list of commits, and performs code analysis on the reproduced code. The information processing apparatus according to claim 3.

5. The code analysis unit performs a test using a test code selected from a plurality of test codes defined for the code. The information processing apparatus according to claim 1.

6. The code analysis unit selects test code using a machine learning model that has learned from the history of tests using modified files and test code. The information processing apparatus according to claim 5.

7. The code analysis unit selects test code by applying rules to the prediction results from the machine learning model. The information processing apparatus according to claim 6.

8. The rule includes at least one of the following: (a) A rule to prioritize the execution of tests that failed last time. (b) A rule to run all tests according to the branch name. (c) A rule to adjust the test selection rate for continuous integrations to a specific branch name. (d) Rules for assigning priority to specific tests The information processing apparatus according to claim 7.

9. Each of the multiple rules has a condition set, The code analysis unit selects test code by applying rules that match the conditions to the prediction results. The information processing apparatus according to claim 7.

10. A difference information acquisition step in which a difference information acquisition unit of a computer acquires difference information between code stored in a first repository and code corresponding to the code stored in a second repository, based on a list of commits that have not yet been pushed to the second repository, The code analysis unit of the computer reproduces the code stored in the first repository from the code stored in the second repository based on the difference information, and performs code analysis on the reproduced code in a code analysis step. An information processing method having

11. A difference information acquisition unit that acquires difference information between code stored in a first repository and code corresponding to the code stored in a second repository, based on a list of commits that have not yet been pushed to the second repository. A code analysis unit that, based on the difference information, reproduces the code stored in the first repository from the code stored in the second repository, and performs code analysis on the reproduced code. A computer program written in a computer-readable format to enable a computer to function.

12. A commit information acquisition device that acquires difference information between the latest code developed in a local environment and the code corresponding to the code that has been pushed to a remote repository, based on a list of commits that are not stored in the remote repository, A code analysis device that, based on the differential information acquired by the commit information acquisition device, reproduces the latest code developed in the local environment from the code already pushed to the remote repository, and performs code analysis on the reproduced code, An information processing system equipped with the following features.

Images