Information processing system

Abstract

To provide an information processing system capable of transversally retrieving source codes of various softwares developed using various development languages, without being restricted by language characteristics.SOLUTION: An information processing system for retrieving a source code according to a meaning attribute includes: an identification processing part for identifying a development language of a processing object file as a processing object, from a code storage part storing the source code; an analysis processing part for associating a code line in the processing object file with the meaning attribute, according to the identified development language of the processing object file, and generating an intermediate file; and an index processing part for performing index processing on the intermediate file, and storing a retrieval index.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing system that can cross-search the source code of various software programs developed using various development languages ​​without being restricted by language characteristics. [Background technology]

[0002] To achieve some kind of processing on a computer, software source code is written using a development language suited to that purpose, and the process is achieved by running it on the computer. For this reason, software is written using source code in a variety of development languages ​​depending on the purpose.

[0003] In today's software development, in order to reduce the burden on software engineers and reduce software development costs, there is a demand to find the desired information from the vast amount of source code repositories made public as open source software (OSS) or from source code resources obtained through previous software development, and then apply that information to develop new source code.

[0004] In order to find the intended information from existing source code and improve software development, it is necessary to search across various types of source code.

[0005] In this case, as described in Non-Patent Document 1, for example, a search command is used to search for and confirm locations containing the relevant keyword. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] MSEdgeTeam et al., "Using the Search Tool to Search Page Source Files," [online], Internet <URL:https: / / learn.microsoft.com / ja-jp / microsoft-edge / devtools-guide-chromium / search / search-tool> Summary of the Invention [Problem to be solved by the invention]

[0007] Conventional search functions such as those described in Non-Patent Document 1 identify and output locations that match the search keywords from the entire search target.

[0008] For example, if the general word "value" is used as a search keyword, the search engine will identify and output "value" occurrences from the source code being searched. As a result, the search results will include "value" with various different meanings, such as "value" as a command in a specific development language, "value" defined as a function, and "value" in normal English as a comment in the source code.

[0009] Even if a software developer is looking for a "value" as a function in the source code, a large number of search results that are unrelated to the search result will be output, and the developer will have to check all of them to remove the noise, which is a burden in itself.

[0010] As mentioned above, software is developed using a variety of development languages ​​depending on the purpose, and for example, development languages ​​such as COBOL may be used for core systems. In addition, there are various other development languages ​​such as C, C++, C#, Java, JavaScript, Python, Perl, PHP, Ruby, VBA, and Delphi.

[0011] For example, if you are redeveloping a system developed in COBOL using C#, and you remember that a similar process to a certain COBOL process is included in a repository of past source code, you can search for specific search keywords in the COBOL and source code repositories and replace the process with the C# development language.

[0012] However, in the past, searches could only be performed within the source code of a specific development language, making it impossible to search across multiple development languages ​​for parts that have the same meaning. [Means for solving the problem]

[0013] In view of the above-mentioned problems, the inventors have invented an information processing system that can perform semantic (meaningful) searches across the source code of various software developed using various development languages, without being restricted by language characteristics.

[0014] A first invention is an information processing system for searching for source code according to semantic attributes, the information processing system having an identification processing unit that identifies the development language of a target file to be processed from a code storage unit that stores source code, an analysis processing unit that associates lines of code in the target file with semantic attributes according to the identified development language of the target file, and generates an intermediate file, and an index processing unit that performs index processing on the intermediate file and stores a search index.

[0015] By configuring the present invention as described above, it is possible to search various source codes developed using various development languages ​​across the board, taking into account their meaning, without being restricted by language characteristics.

[0016] In the above-mentioned invention, the analysis processing unit has common functional semantic extraction rules for each development language, and can be configured as an information processing system that generates the intermediate file by associating lines of code having the functional semantics in the source code written in the identified file to be processed with their semantic attributes based on the extraction rules of the development language in the identified file to be processed.

[0017] The intermediate file can be generated using the process of the present invention, which allows for matching lines of code that share common semantic attributes based on extraction rules for each development language, even if the development languages ​​are different.

[0018] In the above-mentioned invention, the analysis processing unit has common functional semantic extraction rules for each development language, and can be configured as an information processing system that generates the intermediate file by performing tagging processing on lines of code having the functional semantics in the source code written in the identified file to be processed, according to the semantic attributes, based on the extraction rules of the development language in the identified file to be processed.

[0019] With the configuration of the present invention, even if the development languages ​​are different, the same tag can be associated with code lines that have a common semantic attribute based on the extraction rules for each development language.

[0020] In the above-mentioned invention, the analysis processing unit has common functional semantic extraction rules for each development language, and can be configured as an information processing system that generates the intermediate file by associating identification information of a code line having the functional semantic meaning in the source code written in the identified processing target file with the semantic attributes of that code line based on the extraction rules of the development language in the identified processing target file.

[0021] The configuration of the present invention allows identification information and semantic attributes to be associated with code lines, which eliminates the need to write the code itself in the intermediate file, thereby reducing the amount of data in the intermediate file.

[0022] In the above-mentioned invention, the analysis processing unit has common functional semantic extraction rules for each development language, and can be configured as an information processing system that generates the intermediate file by performing tagging processing according to identification information of code lines having the functional semantics in the source code written in the identified processing target file and the semantic attributes of the code lines, based on the extraction rules of the development language in the identified processing target file.

[0023] This configuration allows lines of code to be tagged with identifying information and semantic attributes, eliminating the need for the code itself to be written in the intermediate file, thereby reducing the amount of data in the intermediate file.

[0024] In the above-mentioned invention, the information processing system can be configured as an information processing system having a search processing unit that accepts input of a string to be searched and semantic attributes of the string as search conditions, and outputs some or all of the files to be processed that match the search conditions, and the analysis processing unit links the intermediate file with the files to be processed and stores the linking information in a linking information storage unit, and the search processing unit searches the search index using the search conditions that have been accepted as input, thereby identifying intermediate files that match the search conditions, and outputting some or all of the files to be processed that are linked to the intermediate files.

[0025] In the above-described invention, the information processing system can be configured as an information processing system having a search processing unit that accepts input of a character string to be searched and semantic attributes of the character string as search conditions, and outputs some or all of the target files that match the search conditions, and the analysis processing unit links the intermediate file and its lines of code with the target file and its lines of code, and stores the linking information in a linking information storage unit, and the search processing unit searches the search index using the search conditions that it accepted as input, thereby identifying intermediate files and lines of code that match the search conditions, and outputs lines of code that are the target files that are linked to the intermediate file and that correspond to the code lines of the intermediate file.

[0026] By carrying out the processes of these inventions, it is possible to search for character strings that have specific functional meanings without being restricted by linguistic characteristics.

[0027] The first invention can be realized by executing the program of the present invention on a computer. That is, the information processing program causes a computer to function as an identification processor that identifies the development language of a target file to be processed from a code storage unit that stores source code, an analysis processor that associates lines of code in the target file with semantic attributes according to the identified development language of the target file and generates an intermediate file, and an index processor that performs index processing on the intermediate file and stores a search index. [Effects of the Invention]

[0028] By using the information processing system of the present invention, it is possible to perform semantic (meaningful) searches across the source code of various software developed using various development languages, without being restricted by language characteristics. [Brief explanation of the drawings]

[0029] [Figure 1]1 is a block diagram schematically illustrating an example of a configuration of an information processing system according to the present invention. [Figure 2] FIG. 2 is a block diagram schematically illustrating an example of a hardware configuration of a computer used in the information processing system of the present invention. [Figure 3] 3 is a flowchart showing an example of the overall processing in the information processing system of the present invention. [Figure 4] 1 is a diagram schematically illustrating an example of the overall concept of an information processing system according to the present invention. [Figure 5] FIG. 10 is a diagram schematically illustrating an example of a correspondence relationship between code lines and tags. [Figure 6] FIG. 2 is a diagram illustrating an example of software source code. [Figure 7] FIG. 10 is a diagram illustrating an example of an intermediate file. [Figure 8] FIG. 10 is a diagram illustrating an example of an output screen of a search result. DETAILED DESCRIPTION OF THE INVENTION

[0030] An example of the overall processing functions of an information processing system 1 of the present invention is shown in a block diagram in Figure 1. The information processing system 1 uses a management terminal 2. The management terminal 2 is a computer that causes the information processing system 1 of the present invention to function.

[0031] The management terminal 2 in the information processing system 1 is realized using a computer. An example of the hardware configuration of a computer is shown in Figure 2. The computer has a calculation device 70 such as a CPU that executes the calculation processing of a program, a storage device 71 such as a RAM, hard disk, or SSD that stores information, a display device 72 such as a display that displays information, an input device 73 such as a keyboard or mouse that can input information, and a communication device 74 that sends and receives the processing results of the calculation device 70 and the information stored in the storage device 71 via a network such as the Internet or a LAN.

[0032] If the computer is equipped with a touch panel display, the display device 72 may be integrated with the input device 73. Touch panel displays are often used in portable communication terminals such as tablet computers and smartphones, but are not limited to these.

[0033] The touch panel display is a device that integrates the functions of the display device 72 and the input device 73 in that input can be made directly on the display using a predetermined input device (such as a touch panel pen) or a finger.

[0034] The functions of the various means in the present invention are only logically distinct, and may be physically or practically the same area. The order of the processes in the various means of the present invention may be changed as appropriate. Also, some of the processes may be omitted. For example, the normalization process described below may be omitted. In that case, the process may be performed on image data that has not been normalized.

[0035] The management terminal 2 of the information processing system 1 has a code storage unit 20, an identification processing unit 21, an analysis processing unit 22, an intermediate file storage unit 23, an index processing unit 24, an index storage unit 25, a linking information storage unit 26, and a search processing unit 27.

[0036] The code storage unit 20 stores source code written in various development languages ​​that have been developed in the past and are to be searched. The code storage unit 20 is a so-called source code repository (source code repository). The code storage unit 20 stores source code in various development languages, such as COBOL, C, C++, C#, Java, and JavaScript. Each source code is stored in its own file, and the file extensions are stored in a manner that allows the development language to be identified by the extension, such as ".c" for C, ".cpp" for C++, or ".java" for Java.

[0037] The identification processing unit 21 identifies the development language of each source code stored in the code storage unit 20. The development language is identified based on identification information such as the file extension of the source code.

[0038] The analysis processing unit 22 executes a parsing process for each source code file (processing target file) according to the development language identified by the identification processing unit 21. The parsing process has common functional semantic extraction rules for each development language, and performs a common tagging process for lines of code having specific functional semantics according to the semantic attributes based on the extraction rules for the source code written in the processing target file. Functional semantics refers to predetermined grammar, syntax, etc. that are predefined for executing a specific process in the development language. For example, functional semantic attributes include function definitions, function calls, class definitions, and comments. Any functional semantics may be defined as an extraction rule, and may include not only function definitions, function calls, class definitions, and comments, but also data types, variable definitions, and other arbitrary semantics.

[0039] When tagging is performed using XML, if a code line is determined to define a function, a tag in XML indicating that it is a function is associated and stored. If a code line calls a specific function, a tag indicating that it is a function call is associated and stored. If a code line defines a class, a tag indicating that it is a class is associated and stored. Additionally, if a code line indicates a comment, a tag indicating that it is a comment is associated and stored. The tagging is performed by attaching a tag indicating the start of a specific tagging to the beginning of the code line and a tag indicating the end of the specific tagging to the end of the code line. An example of the correspondence between code lines and tags is shown in FIG. 5. Even if the development languages ​​are different, the same tag can be attached to code lines with the same semantic attribute. Therefore, even if the development languages ​​are different, the tags are common depending on the semantic attribute, so that character strings with the same semantic attribute can be searched for by searching the tags using the search processing unit 27, which will be described later.

[0040] The extraction rules for the parsing process in the analysis processing unit 22 are that the grammar and syntax of function definitions, function calls, class definitions, comments, etc. are defined for each development language, so it is sufficient to set up the system so that the descriptions to be tagged (descriptions corresponding to the grammar and syntax) can be extracted from the source code according to the grammar and syntax of the development language, and then associate tags to correspond to the extracted descriptions.

[0041] The analysis processing unit 22 stores the parsed target file as parsed source code (intermediate file) in the intermediate file storage unit 23. The analysis processing unit 22 also links the target file with the intermediate file, and stores linking information indicating the link in the linking information storage unit 26. The linking information may be a link between identification information such as the file name of the target file and identification information such as the file name of the intermediate file, or may be a correspondence between a line of code in the target file and a corresponding line of code in the intermediate file.

[0042] The intermediate file storage unit 23 stores the intermediate file that has been parsed by the analysis processing unit 22.

[0043] The index processing unit 24 performs index processing for searching the intermediate files stored in the intermediate file storage unit 23. A known index generation process can be adopted for the index processing, and the analysis processing unit 22 stores a search index (semantic search index) including tags according to semantic attributes in the index storage unit 25.

[0044] The index storage unit 25 stores the semantic search index that has been indexed by the index processing unit 24 .

[0045] The linking information storage unit 26 stores linking information indicating the link between the file to be processed and the intermediate file.

[0046] The search processing unit 27 receives input of a character string to be searched and its semantic attribute, searches the semantic search index stored in the index storage unit 25, and passes identification information such as the file name of an intermediate file containing source code that matches the character string and its semantic attribute as a search result. Then, using the identification information of the intermediate file in the search result, the search unit 27 refers to the linking information stored in the linking information storage unit 26 to identify the identification information of the processing target file corresponding to the intermediate file (the source code corresponding to the intermediate file). The processing target file is extracted from the code storage unit 20 and output.

[0047] The search processing unit 27 may also display specific code lines as search results. In this case, the search processing unit 27 accepts input of a character string to be searched and its semantic attribute, searches a semantic search index stored in the index storage unit 25, and passes, as search results, code lines of source code that match the character string and its semantic attribute, as well as identification information for the intermediate file containing those code lines. Then, using the code lines and intermediate file identification information from the search results, the search unit 27 references the linking information stored in the linking information storage unit 26 to identify the identification information for the processing target file corresponding to the intermediate file (the source code corresponding to the intermediate file) and the corresponding code lines in the processing target file. The processing unit 27 extracts the processing target file from the code storage unit 20, and extracts and outputs the portion of the identified code line. [Example]

[0048] Next, an example of processing by the information processing system 1 of the present invention will be described with reference to the flowchart of FIG. 3. Note that software source code is stored in the code storage unit 20, as shown in FIG. 6. FIG. 6(a) shows an example of source code written in the Java language, with the source code file name set to "test.java," and FIG. 6(b) shows an example of source code written in C++, with the source code file name set to "test.cpp." While FIG. 6 shows two source codes as an example, in reality, a much larger number of source codes written in various development languages ​​are stored in the code storage unit 20. For example, hundreds, thousands, or tens of thousands of codes may be stored.

[0049] In addition, the analysis processing unit 22 has extraction rules set so that it can detect code lines that have specific functional meanings in the development language being processed and tag the meanings according to attributes, as shown in Fig. 5. An extraction rule is provided for each development language being processed.

[0050] The identification processing unit 21 extracts a source code file to be processed from the code storage unit 20 (such as a source code repository), and identifies the development language of the source code based on identification information such as the file name (S100). For example, the identification is performed based on the file name extension or a property file associated with the source code file.

[0051] Since the file name of the source code file in Fig. 6(a) is "test.java", the identification processing unit 21 identifies that the development language of the source code in Fig. 6(a) is Java. Also, since the file name of the source code file in Fig. 6(b) is "test.cpp", the identification processing unit 21 identifies that the development language of the source code in Fig. 6(b) is C++.

[0052] The analysis processing unit 22 performs a parsing process on the extracted source code file, tagging lines of code in the source code that have specific functional meanings according to the attributes of those meanings, based on the extraction rules corresponding to the development language identified by the identification processing unit 21 (S110).

[0053] When the source code to be processed is shown in Fig. 6(a), the second line is a code line that defines a function and the third line is a code line that calls the function, so based on the extraction rules of the Java language, the second line is tagged to indicate that it is a function definition and the third line is tagged to indicate that it is a function call, and these are stored as an intermediate file in the intermediate file storage unit 23. An example of the intermediate file of Fig. 6(a) is shown in Fig. 7(a).

[0054] When the source code to be processed is shown in Fig. 6(b), the second line is a function definition, so based on the C++ extraction rules, the second line is tagged to indicate that it is a function definition, and is stored as an intermediate file in the intermediate file storage unit 23. An example of the intermediate file of Fig. 6(b) is shown in Fig. 7(b).

[0055] As described above, intermediate files generated by tagging source code files to be processed stored in code storage unit 20 according to the extraction rules of the development language are stored in intermediate file storage unit 23. The file names of the files to be processed and the file names of the intermediate files are linked and stored as linking information in linking information storage unit 26. Alternatively, the file names of the intermediate files and tagged code lines are linked to the file names of the files to be processed and code lines corresponding to the tagged code lines and stored as linking information in linking information storage unit 26.

[0056] Then, the index storage unit 25 performs index processing on the intermediate files stored in the intermediate file storage unit 23 (S120), and stores a search index (semantic search index) that also includes tags according to semantic attributes in the index storage unit 25.

[0057] By performing the above process, a semantic search index can be built.

[0058] If you want to search for a string with a specific semantic attribute in source code, for example, if you want to search for the string "value" with the semantic attribute of "function definition" in a repository of past source code, you enter the string "value" and its semantic attribute "function definition" on a specified screen.

[0059] Upon receiving this input, the search processing unit 27 searches the semantic search index stored in the index storage unit 25 using the character string "value" and "functiondef," which indicates "function definition" as a semantic attribute, as search criteria, and passes identification information, such as the file name of an intermediate file containing source code that matches the character string and its semantic attribute, as search results. Then, using the identification information of the intermediate file in the search results, the search unit 27 references the linking information stored in the linking information storage unit 26 to identify the identification information of the processing target file corresponding to the intermediate file. The processing target file is extracted from the code storage unit 20 and displayed on the search result output screen. This allows the source code that satisfies the search criteria to be confirmed on the search result output screen.

[0060] Furthermore, when the search processing unit 27 finds source code that satisfies the search conditions and displays its code lines, it may execute the following process. Specifically, the search processing unit 27 accepts the character string "value" and the semantic attribute "functiondef," which indicates "function definition," and uses these as search conditions to search the semantic search index stored in the index storage unit 25. It then passes, as search results, identification information, such as the file name of an intermediate file, containing source code that matches the character string and its semantic attribute. Then, using the identification information of the intermediate file in the search results, it references the linking information stored in the linking information storage unit 26 to identify the identification information of the processing target file corresponding to the intermediate file. The processing target file is extracted from the code storage unit 20 and displayed on the search result output screen. An example of this output screen is shown in FIG. 8. This allows the user to confirm the corresponding code lines of source code that satisfy the search conditions on the search result output screen.

[0061] In the above-described embodiment, the identification processing unit 21 identifies the development language of the target file based on the file extension or property file. However, other methods may be used. For example, the identification process may be performed using a large-scale language model in which a neural network is trained on a large amount of text data. The large-scale language model is a system that performs natural language processing by performing semantic analysis, parameter processing, and the like on instructions entered in a prompt. Therefore, an instruction to identify the source code of the target file and the development language of that source code may be entered in the prompt, and the large-scale language model may return the development language of the source code as an identification result in accordance with the instructions in the prompt. Various models, such as Chat GPT and BERT, may be used as the large-scale language model.

[0062] Furthermore, part or all of the processing in the information processing system 1 of the present invention may be performed using a large-scale language model.

[0063] In the above-mentioned analysis processing unit 22, tagging processing according to semantic attributes is performed as a parsing process, but any other processing that adds information that allows the semantic attributes to be identified, such as adding comments or specified symbols, may be used.

[0064] Furthermore, the intermediate file may include tags, comments, and predetermined symbols added to the lines of code in the target file, as well as information indicating that the lines of code in the target file have semantic attributes, making the lines of code in the target file identifiable. In other words, if a line of code in the target file has a specific semantic attribute, the intermediate file may store identification information for that line of code in the target file (e.g., which line is from the top) in association with the semantic attribute of that line of code. This type of processing can reduce the amount of data in the intermediate file. [Industrial Applicability]

[0065] By using the information processing system 1 of the present invention, it is possible to perform semantic (meaningful) searches across the source code of various software developed using various development languages, without being restricted by language characteristics. [Explanation of symbols]

[0066] 1: Information processing system 2: Management terminal 20: Code memory section 21: Identification processing unit 22: Analysis processing unit 23: Intermediate file storage unit 24: Index processing unit 25: Index storage section 26: Linking information storage unit 27: Search processing section 70: Arithmetic device 71:Storage device 72:Display device 73: Input device 74:Communication equipment

Claims

1. An information processing system for searching source code according to semantic attributes, comprising: The information processing system includes: an identification processing unit that identifies a development language of a file to be processed from a code storage unit that stores source code; an analysis processing unit that associates code lines in the processing target file with semantic attributes in accordance with the development language of the identified processing target file, and generates an intermediate file; an index processing unit that performs index processing on the intermediate file and stores a search index; An information processing system comprising:

2. The analysis processing unit It has common rules for extracting functional meanings for each development language, generating the intermediate file by associating code lines having the functional meaning in the source code written in the file to be processed with semantic attributes based on an extraction rule for the development language in the identified file to be processed; 2. The information processing system according to claim 1, wherein:

3. The analysis processing unit It has common rules for extracting functional meanings for each development language, generating the intermediate file by tagging lines of code having the functional meaning in the source code written in the file to be processed according to the attributes of the meaning, based on an extraction rule for the development language in the identified file to be processed; 3. The information processing system according to claim 2.

4. The analysis processing unit It has common rules for extracting functional meanings for each development language, generating the intermediate file by associating identification information of a code line having the functional meaning in the source code written in the file to be processed with an attribute of the meaning of the code line based on an extraction rule for the development language in the identified file to be processed; 2. The information processing system according to claim 1, wherein:

5. The analysis processing unit It has common rules for extracting functional meanings for each development language, generating the intermediate file by performing tagging processing according to identification information of code lines having functional meaning in the source code written in the file to be processed and semantic attributes of the code lines based on an extraction rule for the development language in the identified file to be processed; 5. The information processing system according to claim 4.

6. The information processing system includes: a search processing unit that receives input of a character string to be searched and a semantic attribute of the character string as search conditions and outputs part or all of the target files that match the search conditions; The analysis processing unit linking the intermediate file with the processing target file, and storing the linking information in a linking information storage unit; The search processing unit searching the search index using the input search conditions, thereby identifying intermediate files that match the search conditions, and outputting part or all of the processing target files linked to the intermediate files; 6. The information processing system according to claim 2, wherein the information processing system is a data processing system.

7. The information processing system includes: a search processing unit that receives input of a character string to be searched and a semantic attribute of the character string as search conditions and outputs part or all of the target files that match the search conditions; The analysis processing unit linking the intermediate file and its code line with the processing target file and its code line, and storing the linking information in a linking information storage unit; The search processing unit searching the search index using the input search conditions, thereby identifying intermediate files and code lines that match the search conditions, and outputting code lines that are the processing target file linked to the intermediate file and that correspond to the code lines of the intermediate file; 6. The information processing system according to claim 2, wherein the information processing system is a data processing system.

8. Computer, an identification processing unit that identifies a development language of a file to be processed from a code storage unit that stores source code; an analysis processing unit that associates code lines in the processing target file with semantic attributes in accordance with the development language of the identified processing target file, and generates an intermediate file; an index processing unit that performs index processing on the intermediate file and stores a search index; An information processing program characterized by causing the program to function as:

Images