A code extraction and analysis method

Abstract

The present invention provides a code extraction and analysis method, comprising the following steps: S1: traversing a code base and copying all codes; S2: extracting code features in a corresponding manner; S3: storing the extracted code files; S4: extracting preset codes from the code files; S5: extracting referenced codes from the code files; S6: assembling results and sending them to a model; S7: calling the model and completing the output; using a large language model instead of manual reading and analysis of codes can greatly improve the efficiency of code reading and analysis; accurately extracting the required key codes, so that the large language model can complete the code analysis work within the range of logical complexity that it is best at; extracting codes across projects and services, so that code analysis is no longer an isolated reading.

Description

Technical Field

[0001] The present invention relates to the technical field of code extraction and analysis, and in particular to a code extraction and analysis method. Background Art

[0002] Large language models (LLMs) are good at handling small-scale code processing tasks. However, in reality, most system structures are complex, often involving dozens or even hundreds of components and millions or even tens of millions of lines of code. If this information is handed over to a large language model for processing, it will be costly and the large language model will often not be able to complete the related work smoothly.

[0003] Usually, during the design and coding phase of the software development process, more than half of the time is spent on reading, analyzing, and debugging old code. If there are many changes in the team members, this loss will become more serious. If these codes are distributed across multiple components, reading such codes is difficult in itself. The functions that need to be adjusted in daily development tasks are usually not complicated, but there are too many old codes to read, which leads to difficulties.

[0004] In order to reduce this part of the loss, the existing technology has begun to combine large language models to realize the processing of this part of the code. However, the difficulty encountered in the existing technology is that the scale of the code base is too large and the content involved is too much, and the large language model cannot process it smoothly. Therefore, it is necessary to extract the code related to the processing task to reduce the code size until the large language model can complete the tasks of reading, debugging and documenting the code of this scale, thereby reducing a lot of time for software developers. Summary of the invention

[0005] In order to solve the defects and shortcomings in the prior art, the present invention provides a code extraction and analysis method.

[0006] The specific scheme provided by the present invention is: A code extraction and analysis method, characterized in that it comprises the following steps: S1: Go through the code base and copy all the code; S2: Use the corresponding method to extract code features; S3: stores the extracted code files; S4: extracting preset codes from the code file; S5: Extract reference code from code files; S6: Assemble the results and send them to the model; S7: Call the model and complete the output.

[0007] As a further preferred embodiment of the present invention, in step S1, a timer is used to actively and regularly trigger a preset script to traverse the code base and copy all codes.

[0008] As a further preferred embodiment of the present invention, in step S1, all codes are copied only from preset branches.

[0009] As a further preferred embodiment of the present invention, in step S2, the extracted code features at least include code features in class information, control layer information, mapping layer information and service layer information.

[0010] As a further preferred embodiment of the present invention, the step S2 includes the following steps: S2.1: Extract code features of class information: traverse all Java files, extract each Java file into a compilation unit, extract class information and interface information from the compilation unit to generate storage files of the corresponding module; S2.2: Extract code features of interface information: determine whether the class information is service layer information from its code features, and then obtain the code features of the corresponding interface through the interface implementation information and the import package corresponding to the interface; S2.3: Extract code features of framework information: first parse the markup language to obtain the preset space in the mapping layer information, and find the interface information corresponding to the mapping layer information through the preset space, then parse the preset tag to obtain the code features corresponding to the preset tag, and introduce code features with preset fields; if the code features corresponding to the preset tag are not obtained, use the preset statements in the markup language to supplement the code features of the mapping layer information.

[0011] As a further preferred embodiment of the present invention, in step S3, the extracted code features are classified and stored in the corresponding modules in a preset file manner.

[0012] As a further preferred implementation of the present invention, in step S4, the preset code at least includes java code, sql code and sql code in markup language.

[0013] As a further preferred embodiment of the present invention, in step S4, When extracting Java code, extract the Java code with the same method name and parameter list as the preset method from the code file; When extracting SQL code, extract the SQL code with preset tags from the code file; When extracting the SQL code in the markup language, first extract the data with the same ID as the interface information from the mapping layer information, then extract the data with preset tags from the data, and introduce the code features with preset fields.

[0014] As a further preferred embodiment of the present invention, the step S5 includes the following steps: S5.1: Delete redundant symbols and unify the code format; S5.2: Extract static reference code using regular expressions; S5.3: Use regular expressions to extract local reference codes.

[0015] As a further preferred embodiment of the present invention, in step S6, when assembling the results, the entry method is placed at the front, and the codes are sorted according to the call chain and the call frequency.

[0016] Compared with the prior art, the present invention can achieve the following technical effects: 1) The present invention provides a code extraction and analysis method, which uses a large language model to replace manual reading and analysis of codes, and can greatly improve the efficiency of code reading and analysis.

[0017] 2) The present invention provides a code extraction and analysis method, which accurately extracts the required key codes, so that the large language model can complete the code analysis work within the range of logical complexity that it is best at.

[0018] 3) The present invention provides a code extraction and analysis method, which extracts codes across projects and services, so that code analysis is no longer an isolated reading.

[0019] 4) The present invention provides a code extraction and analysis method with strong expansion capability. Data status can be added later. By extracting data in a configuration state, problem analysis is more accurate, and other types of large language models can be used and replaced at any time.

[0020] 5) The present invention provides a code extraction and analysis method with low cost. The task can be successfully completed by using only cloud services without the need for special training models, thus saving time and effort.

[0021] 6) The present invention provides a code extraction and analysis method, which uses corresponding different methods to extract code features of different information, and then classifies and stores them in the form of different code files, thereby improving the extraction efficiency and accuracy while facilitating the subsequent calling process of the code file; at the same time, by using different methods to extract the required different codes, the effectiveness and reliability of code extraction and subsequent code analysis are further improved, and the code analysis efficiency and analysis accuracy are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Shown is a flow chart of the method steps provided by the present invention; Figure 2 Shown is a logic flow chart of code extraction of the present invention; Figure 3 The figure shows a schematic diagram of the code for extracting class information and interface information in step S2.1 of the present invention; Figure 4 The schematic diagram of extracting only some types of codes that affect code extraction in step S2.1 of the present invention is shown; Figure 5 The figure shows a schematic diagram of the header definition code of a typical service code in step S2.2 of the present invention; Figure 6 The schematic diagram of the code for extracting the code features in the interface information in step S2.2 of the present invention is shown as follows: Figure 7 The code diagram of the markup language xml of the typical framework information mybatis in step S2.3 of the present invention is shown; Figure 8 It is a schematic diagram of code for extracting code features in framework information in step S2.3 of the present invention; Fig. 9 The diagram shows a code diagram of the SQL code written in the declaration of the interface class in step S2.3 of the present invention; Fig.10 The figure shows a schematic diagram of a code for extracting Java code in step S4 of the present invention; Fig.11 The figure shows a code schematic diagram of extracting the SQL code in step S4 of the present invention; Fig.12 The figure shows a schematic diagram of a code for extracting the SQL code in the markup language in step S4 of the present invention; Fig.13 It is a code schematic diagram of extracting static reference code by using regular expression in step S5.2 of the present invention; Fig.14 It is a code schematic diagram of extracting local reference code by using regular expression in step S5.3 of the present invention; Fig.15 Shown is a schematic diagram of the final output result of the present invention. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] In the description of the present invention, it should be noted that the terms "upper", "lower", "inner", "outer", "front end", "rear end", "two ends", "one end", "the other end" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0025] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "provided with", "connected", etc. should be understood in a broad sense. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0026] [First embodiment] Since the core businesses of the current springboot framework and mybatis framework are concentrated in sql code and java code, take the process of obtaining sql code and java code from the framework that includes both springboot and mybatis as an example. Figure 1 The first embodiment of the present invention is shown, which provides a code extraction and analysis method, comprising the following steps: S1: Traverse the code base and copy all the code; e.g. Figure 2 As shown, a timer can be used to actively and regularly trigger a preset script. In this embodiment, the code base can be selected from, for example, a git database, and the preset script corresponds to a git script. The triggering frequency can be set according to actual needs, such as actively triggering once every day or every week, etc., to traverse the code base and copy all the codes.

[0027] It is worth noting that in this embodiment, all codes are copied only from a preset branch, which can be set to, for example, a master branch, because the master branch is the main branch, which is used for all subsequent work, and therefore can be set to copy all codes only from the master branch.

[0028] S2: extracting code features in a corresponding manner; the extracted code features at least include code features in class information, controller information of the control layer, mapper information of the mapping layer, and service information of the service layer.

[0029] Since the target code is a Java framework project based on Springboot for analysis, although the basic component unit of Java is class, Springboot also adds many other unit concepts, such as controller layer, service layer, component layer Component Api, etc. They can essentially be regarded as a class. Therefore, when designing the code extraction logic, they are uniformly processed in a class manner.

[0030] According to this classification, there are still many unit concepts, such as Controller Rest, Controller Request, Mapping Get, Mapping Post and other types. Based on this, they are simplified into the following three types: 1) Single class type: such as static method, Component method, etc.; 2) Interface type: such as API Service, etc. 3) Framework type: for example, the interface type and the type of XML implementation in mybatis; Those skilled in the art know that, for the existing code types such as aspects and declarations, since they are usually codes related to the code framework, they are relatively rarely used in business and usually do not change the logical meaning of the business code, so they are not considered in this embodiment.

[0031] The code features extracted in this embodiment are essentially the code features corresponding to the above-mentioned type information; Therefore, in step S2, the following steps are included: S2.1: Extracting code features of class information: Since the basic unit of Java code is class, the com.github.javaparser class library is used to traverse all Java files and extract each Java file into a compilation unit, namely CompilationUnit; using compilation units can effectively reduce complexity. After entering the compilation unit, only the core code can be extracted according to the compilation rules, while ignoring a large number of other irrelevant details. For example, in this embodiment, since there is a lot of information in the compilation unit, only the class information class and interface information interface need to be extracted, such as Figure 3 As shown, to generate the storage file of the corresponding module; It is worth noting that in order to avoid the cumbersome process and time consumption that may result from processing numerous node types, in this embodiment, only some types that affect code extraction are extracted for processing, such as Figure 4As shown, the remaining types that do not affect code extraction are handled by the larger language model that is more proficient in it.

[0032] S2.2: Extract code features of interface information: determine whether the class information is service layer information from its code features, and then obtain the code features of the corresponding interface through the interface implementation information and the import package corresponding to the interface; After extracting the interface and class, the class declaration will include the service layer Service label, such as Figure 5 As shown, by judging whether the code feature of the class information contains the service layer Service label, it can be determined whether it is the service layer service information. Since the service layer information can implement one or more interfaces, and the interface must be imported through the corresponding import package, the code feature of the corresponding interface can be obtained through the interface implementation information and the import package corresponding to the interface, such as Figure 6 As shown; S2.3: Extract code features of framework information: Because the typical framework information mybatis markup language xml has a clear format, for example: Figure 7 As shown, it must be = <mapper namespoace="”xxxxxx.xxx.x.xx”">This will facilitate the identification of whether it is the mapping layer mapper information, and it must contain <insert> 、 <delete>etc. Therefore, we first parse the markup language XML to obtain the preset space namespace in the mapping layer mapper information, and then find the interface information corresponding to the mapping layer mapper information through the preset space namespace, and then parse the preset tags (for example <insert> 、 <delete>etc.) to obtain the code features corresponding to the preset tag, and introduce the code features with preset fields, such as field include. If there is an include in the middle, you need to further introduce the sql code entity of the include, such as Figure 8 As shown; If the code feature corresponding to the preset tag is not obtained, the preset statement in the markup language is used to supplement the code feature of the mapping layer information; this is because the code of the mapping layer mapper information is special, because the SQL code can be written in the markup language XML or in the declaration of the interface class, such as Fig. 9 As shown, therefore, correspondingly, the logic of the code processing here is to parse the markup language XML first, and then parse the interface class. If the interface class code does not contain <insert> 、 <update>If there are preset tags, the sql code in the markup language xml is used to supplement them to synthesize a new mapping layer mapper object.

[0033] S3: Store the extracted code files; classify and store the extracted code features in the corresponding modules in a preset file format, such as a json file; the classification storage method needs to correspond to the extracted code feature category, for example, the class information is stored in the json file of Allclassbean, the controller information of the control layer is stored in the json file of Allcontrollerbean, the mapper information of the mapping layer is stored in the json file of Allmapperbean, and the service information of the service layer is stored in the json file of Allservicebean, so as to facilitate classification storage and subsequent calling process, such as Figure 2 As shown; S4: extracting preset codes from the code file; in this embodiment, the preset codes include at least java codes, sql codes and sql codes in the markup language; wherein, When extracting Java code, you can use the abstract syntax tree method to extract Java code with the same preset method name and parameter list from the code file, such as Fig.10 As shown; When extracting SQL code, extract the SQL code with preset tags from the code file; the preset tags include <insert> 、 <update>Etc., the code containing these preset tags is considered to be SQL code, such as Fig.11 As shown; When extracting SQL code in the markup language, first extract the data with the same ID as the interface information from the mapping layer mapper information, and then extract the data with the preset tag (such as <insert> 、 <delete>etc.), and introduce code features with preset fields, such as field include. If there is an include in the middle, it is necessary to further extract the SQL code associated with the include, such as Fig.12 shown.

[0034] S5: extracting referenced code from the code file, by finding the location of other code referenced in the code, such as package name, class name, method name, etc., to find the referenced code in the extracted code; specifically including the following steps: S5.1: Delete redundant symbols and unify the code format; for example, line breaks, \t, and redundant spaces in the code will affect the subsequent regular expression extraction. First remove the symbols that do not affect the code semantics but affect the regular expression extraction, so as to further ensure the smooth and accurate extraction of the subsequent regular expression; S5.2: Extract static reference code using regular expressions; Regular Expression is a text pattern matching tool that searches, matches, and operates text by defining specific patterns. For example, in this embodiment, regular expressions can be used to extract codes of this type: String companyName=companyService.companyName(cid); This type of code represents static reference code, such as service, component, api reference, etc. Fig.13 As shown; S5.3: Similarly, regular expressions can also be used to extract local reference codes, such as Fig.14 shown.

[0035] S6: Assemble the results and send them to the model; after all the relevant codes are extracted and before they are handed over to the large language model, the codes need to be sorted and sorted. Therefore, when assembling the results, the entry method needs to be put in the front, and the code needs to be sorted according to the call chain and the call frequency. This is because the entry represents the entry of the business, so the output information is more in line with the current business goals. According to actual needs, the service method can also be put in the front, but this sorting method is more used for correlation analysis. Correspondingly, the prompt words handed over to the large language model will also become similar to "analyze this service, if it makes xxx changes, will it cause which codes to need to be modified", and the requirements of the code extraction and analysis method in the current embodiment are: to be able to answer what the business logic of this interface is, so the entry method needs to be put in the front.

[0036] S7: Call the model and complete the output. The code extraction and analysis results implemented by the method of this embodiment are as follows: Fig.15 As shown, using a large language model instead of manual code reading and analysis can greatly improve the efficiency of code reading and analysis.

[0037] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.< / delete> < / insert> < / update> < / insert> < / update> < / insert> < / delete> < / insert> < / delete> < / insert> < / mapper>

Claims

1. A code extraction and analysis method, characterized in that: The following steps are involved: S1: Go through the code base and copy all the code; S2: Use the corresponding method to extract code features; S3: stores the extracted code files; S4: extracting preset codes from the code file; S5: Extract reference code from code files; S6: Assemble the results and send them to the model; S7: Call the model and complete the output.

2. A code extraction and analysis method according to claim 1, characterized in that: In the step S1, a timer is used to actively and regularly trigger a preset script to traverse the code base and copy all codes.

3. A code extraction and analysis method according to claim 1, characterized in that: In step S1, all codes are copied only from the preset branch.

4. A code extraction and analysis method according to claim 1, characterized in that: In the step S2, the extracted code features at least include code features in class information, control layer information, mapping layer information and service layer information.

5. A code extraction and analysis method according to claim 4, characterized in that: The step S2 includes the following steps: S2.1: Extract code features of class information: traverse all Java files, extract each Java file into a compilation unit, extract class information and interface information from the compilation unit to generate storage files of the corresponding module; S2.2: Extract code features of interface information: determine whether the class information is service layer information from its code features, and then obtain the code features of the corresponding interface through the interface implementation information and the import package corresponding to the interface; S2.3: Extract code features of framework information: first parse the markup language to obtain the preset space in the mapping layer information, and find the interface information corresponding to the mapping layer information through the preset space, then parse the preset tag to obtain the code features corresponding to the preset tag, and introduce code features with preset fields; if the code features corresponding to the preset tag are not obtained, use the preset statements in the markup language to supplement the code features of the mapping layer information.

6. A code extraction and analysis method according to claim 1, characterized in that: In the step S3, the extracted code features are classified and stored in the corresponding module in a preset file manner.

7. A code extraction and analysis method according to claim 5, characterized in that: In the step S4, the preset code at least includes java code, sql code and sql code in markup language.

8. A code extraction and analysis method according to claim 7, characterized in that: In the step S4, When extracting Java code, extract the Java code with the same method name and parameter list as the preset method from the code file; When extracting SQL code, extract the SQL code with preset tags from the code file; When extracting the SQL code in the markup language, first extract the data with the same ID as the interface information from the mapping layer information, then extract the data with preset tags from the data, and introduce the code features with preset fields.

9. A code extraction and analysis method according to claim 1, characterized in that: In the step S5, The following steps are involved: S5.1: Delete redundant symbols and unify the code format; S5.2: Extract static reference code using regular expressions; S5.3: Use regular expressions to extract local reference codes.

10. A code extraction and analysis method according to claim 1, characterized in that: In step S6, when assembling the results, the entry method is placed at the front, and the codes are sorted according to the call chain and the call frequency.

Images