Semantic service performance prediction method based on component and scale characteristics
Through the method based on component and scale features, a semantic service database and performance testing system are built, key impact features are determined and performance prediction models are established, which solves the problem of poor accuracy of semantic service performance prediction, and achieves fast and low-cost semantic service performance prediction .
Patent Information
- Application Number
- CN202411045698.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-11-19
AI Technical Summary
In the prior art, semantic service performance prediction is poor, and time cost and physical resource consumption is high, and methods can predict semantic service performance in all cases through a small number of tests.
Using a semantic service performance prediction method based on component and scale features, by building a semantic service database and performance testing system, the key impact characteristics of semantic atom components are determined, and the performance prediction model is established to achieve rapid prediction of semantic service performance.
It reduces the time cost and physical resource consumption of semantic service performance prediction, and provides a general performance prediction method, which is more adaptable and more accurate, and can quickly predict semantic service performance.
Smart Images

Figure CN118981432A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of semantic service performance prediction, and in particular to a semantic service performance prediction method based on component and scale characteristics. Background Art
[0002] A Domain-Specific Language (DSL) is a programming language designed specifically for a specific domain or target problem set. Unlike a General Purpose Language (GPL), a DSL is tailored to the needs of a specific application domain, making it more expressive and efficient in tasks within that domain. Due to these characteristics, DSLs have been successfully applied in improving software development efficiency and software quality in various fields, such as embedded systems, robotics, multimedia, multi-agent systems, home automation, distributed system protocols, Industry 4.0 aviation operating systems, machine learning, etc. DSLs are also used to enhance the capabilities of activities in the software development process, such as ensuring software quality or assisting code analysis, code debugging, formal verification, etc.
[0003] After the DSL semantic service is developed, the performance of the semantic service needs to be tested. Generally speaking, the existing DSL file is used as the test input, and the performance of the semantic service is tested by actually running the semantic service. However, this actual testing method often requires a lot of resources and time to build a test environment and conduct repeated tests, and the test results are difficult to cover all situations.
[0004] The research on DSL semantic services is basically the research on service classification, and there is no research on its performance testing or prediction. If the prediction target is not considered as semantic services, the most relevant program running time prediction methods for the present invention are static analysis methods and machine learning-based methods. For static analysis methods, traditional static analysis methods predict the execution time of a program by analyzing the time complexity of the algorithm. First, the maximum input size of the algorithm is measured, such as the length of the array, and then the number of executions of the statements in the algorithm in the worst case is analyzed, and the upper limit of the algorithm time complexity is determined using O notation. This analysis method is often used to analyze the time complexity of some common algorithms (such as sorting algorithms) and operations on data structures (such as insertion and deletion in red-black trees). However, this method has limitations. On the one hand, it can predict the change of algorithm execution time with the increase of input size, but cannot accurately predict the specific time. On the other hand, for complex programs, it becomes more difficult to directly analyze the running time. For machine learning methods, machine learning-based methods do not rely on analyzing specific code snippets. Instead, they identify useful features in the program and then use these features together with test data to train performance prediction algorithms. Machine learning methods do not need to extract detailed information about code structure or behavior, nor do they need to understand the computing resource platform. They focus on extracting abstract code features that affect program performance, thereby saving analysis costs. However, machine learning methods need to ensure the quantity and quality of training data, otherwise the prediction effect is difficult to guarantee. Therefore, a general method for predicting semantic service performance is needed, which can predict semantic service performance in all cases through a small number of tests. Summary of the invention
[0005] In view of the above analysis, an embodiment of the present invention aims to provide a semantic service performance prediction method based on component and scale characteristics, so as to solve the problems in the prior art of poor semantic service performance prediction accuracy, high time cost and high physical resource consumption.
[0006] The embodiment of the present invention provides a method for predicting the performance of a semantic service based on component and scale characteristics, including: S1: constructing a semantic service database, wherein the semantic service database stores various types of semantic services and semantic atomic components contained in the various types of semantic services and their call times; S2: constructing a semantic service performance testing system, wherein the semantic service performance testing system is used to obtain the execution time of the semantic services and the semantic atomic components; S3: constructing a code snippet, generating a code file set based on the code snippet, and using the semantic service performance testing system to obtain the first execution time of the code file set, wherein the first execution time includes the execution time of each semantic atomic component; S4: based on the code file set and the first execution time, the semantic service performance testing system is used to obtain the execution time of the semantic service and the semantic atomic components. A permutation importance algorithm is used to determine the key influencing features corresponding to each semantic atomic component; S5: a test file set of each key influencing feature is established based on the code snippet, and a second execution time of each test file set is obtained based on the semantic service performance testing system, wherein the second execution time includes the execution time of each semantic atomic component, and a performance prediction model of each semantic atomic component is obtained based on each test file set and its corresponding second execution time; S6: the semantic service to be tested is obtained, and the semantic atomic components corresponding to the semantic service to be tested and their execution times are obtained based on the semantic service database query, and the performance prediction result of the semantic service is calculated based on the performance prediction model of each semantic atomic component and the execution times.
[0007] Based on the further improvement of the above method, the semantic service performance testing system is implemented based on vscode / test-web, including a server and a client. The server is used to write and store semantic services, and the client is used to request semantic services and fill in parameters related to semantic services.
[0008] Based on the further improvement of the above method, the semantic service includes: semantic tokens, document highlighting, file symbols, naming, workspace symbols, go to declaration, go to definition, go to type definition, go to implementation, find reference, call hierarchy, type hierarchy, document link, hover, inline value embedding prompt, signature help, scope filter completion, symbol type filter completion, function signature completion, existence check, type consistency check, accessibility check; the semantic atomic components include: construction, positioning, search, traversal; the number of calls is the number of executions of each semantic atomic component in the semantic service.
[0009] Based on the further improvement of the above method, the constructing code snippets and generating a code file set based on the code snippets include: S31: constructing DSL code snippets with different cyclomatic complexities; S32: selecting any code feature as a variable, setting the minimum value, maximum value and number of code files in the code file set of the code feature, and calculating the generation step length based on the number of code files, the minimum value and maximum value of the code feature, based on the minimum value and the generation step length, using the code feature as a variable, and using the code snippet to generate a group of code files that vary according to the generation step length, thereby obtaining a code file set.
[0010] Based on the further improvement of the above method, the key influencing features corresponding to each semantic atomic component are determined by using a permutation importance algorithm based on the code file set and the first execution time, including: S41: constructing a first data sample set based on the code file set, each sample in the first data sample set includes a semantic atomic component corresponding to a code file, a feature value of each code feature and an execution time of each semantic atomic component, and dividing the first data sample set into a training set and a test set according to a certain ratio; S42: constructing a machine learning model; S43: selecting any semantic atomic component from the unselected semantic atomic components, taking the feature value of each code feature of each code file in the training set as an input data, and the execution time of the semantic atomic component in the code file as a label, to train the machine learning model; S44: after the training is completed, the feature value of each code feature of each code file in the test set is input as input data to the machine learning model, and the determination coefficient is calculated based on the output predicted execution time and the execution time of the semantic atomic component in the test set. S45: Select a code feature from the unselected code features, disrupt the original correspondence between the feature value of the code feature and the code file in each code file in the test set, and then input the test set into the machine learning model, and calculate the execution time of the semantic atomic component in the test set based on the output predicted execution time by As the importance score of the code feature; S46: Repeat step S45 until each code feature in the test set is calculated, and take the code feature corresponding to the maximum importance score as the key influencing feature of the current semantic atomic component; S47: Repeat S43-S46 until all semantic atomic components obtain the corresponding key influencing features.
[0011] Based on further improvement of the above method, the code features include: number of code lines, number of defined identifiers, number of occurring identifiers, cyclomatic complexity, number of nodes, degree, and depth.
[0012] Based on the further improvement of the above method, the establishment of a test file set based on the code snippet and the key influencing feature includes: S51: selecting a feature from the key influencing features, setting the minimum value and maximum value of the feature in the test file set, and setting the number of files in the test file set, calculating a generation step based on the number of files, the minimum value and the maximum value, based on the minimum value and the generation step, taking the key influencing feature as a variable, using the code snippet to generate a group of test files that change according to the generation step, thereby forming a test file set; S52: counting the minimum value and maximum value of the feature values of other key influencing features of the test file set; S53: selecting a feature from the key influencing features that have not been selected, and calculating a generation step based on the number of files, the minimum value and the maximum value of the feature, based on the minimum value and the generation step of the feature, taking the feature as a variable, using the code snippet to generate a group of test files that change according to the generation step, thereby forming a test file set; S54: repeating S53 until all key influencing features have established corresponding test file sets.
[0013] Based on the further improvement of the above method, the performance prediction model of each semantic atomic component is obtained based on the test file sets and their corresponding second execution time, including: constructing a performance prediction model for each semantic atomic component; obtaining a performance prediction model for each semantic atomic component by the following method: obtaining the characteristic values of the key influencing features corresponding to the semantic atomic component in each test file based on the test files in the test file set of the semantic atomic component, and obtaining the execution time of the semantic atomic component corresponding to each test file in the second execution time; fitting the characteristic values and execution times corresponding to each code file to obtain the fitting coefficients of the performance prediction model, thereby obtaining the performance prediction model of the semantic atomic component.
[0014] Based on the further improvement of the above method, the performance prediction model of the semantic atomic component is constructed as follows:
[0015] SSC i (δ k )=k*δ k +c,
[0016] Among them, SSC i (δ k ) is the key influencing feature δ k The corresponding prediction performance of the i-th semantic atomic component, k and c are the fitting coefficients.
[0017] Based on the further improvement of the above method, the performance prediction result of the semantic service is obtained through the following formula:
[0018]
[0019] Among them, SSP is semantic service performance, SSC i is the prediction performance of the i-th semantic atomic component, times i is the execution times of the i-th semantic atomic component, i=1, 2, 3, ..., n.
[0020] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:
[0021] 1. The present invention provides a semantic service performance prediction method based on component and scale characteristics, constructs the relationship between semantic services and semantic atomic components, and the relationship between semantic atomic components and key influencing features, uses key influencing features as evaluation indicators of semantic atomic component service performance, and reduces the time cost and physical resource consumption of semantic service performance prediction.
[0022] 2. The present invention provides a semantic service performance prediction method based on component and scale characteristics, and provides a general method for semantic service performance prediction. For the semantic service to be tested, the number of component executions is obtained by querying the semantic service database, and combined with the performance prediction model of the semantic atomic component, the semantic service performance can be quickly predicted.
[0023] 3. The present invention provides a semantic service performance prediction method based on component and scale characteristics. The parameters of the performance prediction model of the semantic atomic component are obtained based on a test file set constructed based on DSL code snippets of different cyclomatic complexities. The test file set covers different types and numbers of code features. Therefore, the performance prediction model of the present invention has wider adaptability, higher accuracy and stronger versatility.
[0024] In the present invention, the above-mentioned technical solutions can also be combined with each other to achieve more preferred combination solutions. Other features and advantages of the present invention will be described in the subsequent description, and some advantages can become obvious from the description, or can be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] The accompanying drawings are only used for the purpose of illustrating specific embodiments and are not to be considered as limiting the present invention. In the entire drawings, the same reference symbols represent the same components;
[0026] Figure 1 A schematic diagram of a semantic service performance prediction method based on component and scale characteristics;
[0027] Figure 2 Schematic diagram of the correspondence between the feature values of key influencing features of semantic atomic components and the execution time. DETAILED DESCRIPTION
[0028] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.
[0029] A specific embodiment of the present invention discloses a semantic service performance prediction method based on component and scale characteristics, such as Figure 1 As shown, including:
[0030] S1: constructing a semantic service database, wherein the semantic service database stores various semantic services and semantic atomic components contained in the various semantic services and their call times.
[0031] The semantic services include: semantic tokens, document highlighting, file symbols, naming, workspace symbols, go to declaration, go to definition, go to type definition, go to implementation, find reference, call hierarchy, type hierarchy, document link, hover, inline value mosaic prompt, Moniker, signature help, scope filter completion, symbol type filter completion, function signature completion, existence check, type consistency check, accessibility check, duplicate definition check; the semantic atomic components include: construction, positioning, search, traversal; the number of calls is the number of executions of each semantic atomic component in the semantic service.
[0032] In particular, the LSP specification version 3.17 divides services into five categories: lifecycle messages, services related to the server lifecycle, including operations when the client starts and shuts down the server; document synchronization, services that synchronize the document state between the client and the server; language features, services related to code understanding and writing; workspace features, services that operate symbols and files within the workspace; window features, services that display specific message notifications on the client. Since the language services in the categories of lifecycle messages, document synchronization, and window features do not require specific semantic information, such as symbol types, in their implementation, they are not considered semantic services. The services in the language features and workspace features categories were reviewed and 21 semantic services were finally identified. Of these 21 semantic services, 20 belong to language features. This is because the language features category provides actual intelligence in LSP compared to other categories. These services that provide intelligence usually require semantic information, so they usually belong to semantic services.
[0033] Even within the language feature category, different semantic services may have significant differences in functionality. For example, the definition jump service needs to resolve the definition location of a symbol at a given text document location, while the rename service needs to rename a symbol within the workspace scope. Of course, there may also be semantic services with similar functionality. For example, the difference between the definition jump and the find reference service is that the former resolves the definition location of a symbol, while the latter resolves the reference location of a symbol. Semantic services with significant functional differences may have significant differences in their execution processes, indicating that their performance may be significantly different. On the contrary, semantic services with minor functional differences usually have similar execution processes, indicating that their performance may be very similar. Therefore, in order to clarify the boundaries of semantic services, the semantic services in the LSP specification are reclassified, and semantic services with similar execution processes are classified into the same group, resulting in 5 categories of semantic services: Highlight: These services provide highlighting support for symbols in documents, such as highlighting different types of symbols with different colors. Symbol Operation: These services are responsible for handling symbol-related operations, such as finding and modifying symbols. Code Navigation: These services help users understand and navigate code. It includes finding and jumping to the definition or reference of a symbol, browsing other information related to the symbol, such as symbol annotations and parameter information of function symbols. Code Completion: These services provide code completion suggestions based on the contextual information around the cursor position. In practice, common code completion includes scope filtering completion, symbol type filtering completion, and function signature completion. Code Diagnostic: These services provide semantic error diagnostic information for the code. The LSP specification does not describe the classification of error diagnostic information. In practice, determining the semantic errors to be checked depends on the programming language. For example, in variable assignment and operation, weakly typed languages like JavaScript have loose type checking and allow implicit conversions between different data types, while strongly typed languages like Java require type checking, and if the types are inconsistent, type errors will occur. In order to define the boundaries of semantic error diagnostic services, this invention summarizes 4 common semantic error categories: existence check, type consistency check, accessibility check, and duplicate definition check.
[0034] In order to identify the components in the semantic service and predict its performance, it is first necessary to analyze the execution process of the semantic service, including: parsing the file to obtain an abstract representation, usually an abstract syntax tree (AST); traversing the abstract representation to collect necessary information; processing the collected information to implement the semantic service. Assuming that the result of the file parsing in the first step is an AST, a symbol table is used in the second step to store the information. If the file has not been modified, a single AST and symbol table can be used to implement various semantic services; if the file has been modified, the file needs to be re-parsed to generate a new AST and rebuild the symbol table. Therefore, parsing the file and building the symbol table are tightly coupled operations. In other words, the first and second steps can be combined into one step. In the third step of implementing the semantic service, it is usually necessary to first determine the symbol pointed to by the cursor position, then find the symbol in the symbol table according to its name, and collect other necessary information by accessing the AST. Finally, the collected information is used to implement the semantic service. Based on the above process of implementing semantic services, the present invention finally defines four semantic components: Construct, which involves parsing the file and generating AST, and then building a symbol storage structure based on AST, such as a symbol table or a semantic tree; Locate, which requires obtaining the corresponding AST according to the document identifier sent by the client, and locating the corresponding identifier in the symbol storage structure according to the cursor position information; Search, which requires searching for instances of symbols in the symbol storage structure according to the symbol name; Traverse, which requires different strategies or algorithms to access the symbol storage structure in different semantic services. Taking the tree structure as an example, in the "Rename" service, graph traversal algorithms such as depth-first search can be used to rename symbols in the workspace. Similarly, in the "Go to Definition" service, a bottom-up strategy can be used to traverse the tree and obtain the definition of the symbol.
[0035] The execution process of the semantic service is composed of different components, which can appear individually or multiple times. In particular, when building a symbol table, a semantic error check is usually performed at the same time, which means that the "diagnosis" service is implemented simultaneously with the construction of the symbol table. In addition, if the file remains unchanged, this symbol table can be used to implement other semantic services. Therefore, the present invention binds the "construction" component to the "diagnosis" service, and other semantic services no longer include this component.
[0036] Table 1 shows each semantic service and its corresponding semantic atomic component. Analyzing the composition of semantic services not only helps to better understand semantic services from the developer's perspective, but also provides support for subsequent experiments to predict the performance of semantic services.
[0037] Table 1. Semantic services and their corresponding semantic atomic components
[0038]
[0039]
[0040] The number of calls is obtained by analyzing the principle of the semantic service execution process and experimentally verifying it. Schematically, for the number of calls to the search component Search in the highlight service semantic tokens service, it is speculated that it is related to the number of identifiers OCC that appear in the file. Then, the number of component calls to the service is calculated experimentally, and it is verified that the number of calls to the search component is equal to OCC. Table 2 shows a schematic diagram of the number of calls to some semantic atomic components in the semantic service.
[0041] Table 2 Call times of some semantic atomic components in semantic services
[0042]
[0043]
[0044] Preferably, except for the code diagnosis service, other services need to call the positioning component once, that is, to get the symbol pointed to by the cursor. Taking the semantictokens service in semantic highlighting as an example, this service needs to mark all identifiers with colors, where different types of identifiers require different colors. Therefore, in theoretical analysis, each time an identifier is encountered, its type needs to be searched, that is, the number of calls to the search component is equal to the number of identifiers that appear, and the whole process only needs to traverse the AST once to complete; taking the rename service in symbol operation as an example, this service needs to uniformly rename a symbol in the file. This process requires finding the definition of the symbol and then traversing the AST downward once. Therefore, theoretical analysis requires calling one search component and one traversal component. Taking the code diagnosis service as an example, in the statement x=max(1, y), it is necessary to check whether x, max, and y are defined. Then check whether the return types of x and max functions are consistent, and whether the parameter types of y and max are consistent. Check whether x, max, and y are accessible, etc. Therefore, the entire diagnostic service theoretically requires multiple table lookups and multiple AST traversals. However, if only simple diagnosis is considered, such as only existence check, it is only necessary to traverse the AST once and search for each identifier to see if it is defined.
[0045] S2: Construct a semantic service performance test system, where the semantic service performance test system is used to obtain the execution time of semantic services and semantic atomic components.
[0046] The semantic service performance test system is implemented based on vscode / test-web, including a server and a client. The server is used to write and store semantic services, and the client is used to request semantic services and fill in parameters related to semantic services. Code is added to the server to calculate the execution time of semantic services and semantic atomic components.
[0047] Generally, the semantic service time perceived by users consists of three components: the transmission time between the client and the server, the processing time of the server, and the rendering time of the IDE or editor. Since the transmission time delay depends on the network environment, and the rendering time depends on the specific IDE or editor, the semantic service performance in this application refers to the time it takes for the server to process and return a response after receiving the client request. The execution time of a semantic atomic component refers to the time from when the semantic atomic component is called to when it ends.
[0048] S3: construct a code snippet, generate a code file set based on the code snippet, and use the semantic service performance testing system to obtain a first execution time of the code file set, where the first execution time includes the execution time of each semantic atomic component.
[0049] The constructing code snippets and generating a code file set based on the code snippets include: S31: constructing DSL code snippets with different cyclomatic complexities; S32: selecting any code feature as a variable, setting the minimum value, maximum value and number of code files in the code file set of the code feature, calculating a generation step based on the number of code files, the minimum value and maximum value of the code feature, based on the minimum value and the generation step, using the code feature as a variable, and using the code snippet to generate a group of code files that vary according to the generation step, thereby obtaining a code file set.
[0050] Referring to Table 3, the code features include: number of code lines, number of defined identifiers, number of appearing identifiers, cyclomatic complexity, number of nodes, degree, and depth.
[0051] Table 3 Code features
[0052] feature Abbreviation describe Lines of code LOC Number of lines in code file excluding comments Define the number of identifiers DEF The number of identifiers defined in the code file Number of occurrences of identifier OCC The number of occurrences of the identifier in the code file Cyclomatic complexity CYC Cyclomatic complexity of code files Number of nodes in AST NODE The number of nodes in the abstract syntax tree generated by parsing the code file Degree of AST DEGREE The degree of the abstract syntax tree generated by parsing the code file Depth of AST DEPTH The depth of the abstract syntax tree generated by parsing the code file
[0053] Specifically, the number of lines of code (LOC). Generally speaking, the more lines of code there are, the larger the code size is, which will lead to poor performance of semantic services; the number of identifiers. In a program, identifiers are usually used to represent variables, functions, etc. Although Halstead proposed a method of using operators and operands to measure the length of a program, this method is not suitable for semantic services because keywords do not directly affect performance. Therefore, the present invention uses two other features to replace Halstead's method: the number of defined identifiers and the number of identifiers that appear. These two features are related to the "build" and "search" components. As the number of defined identifiers increases, the number of symbols stored in the symbol table also increases, thereby increasing the time required to run these two components. In addition, some semantic services (such as "semantic tagging") need to obtain information about all identifiers that appear in the document through the "search" component. Therefore, as the number of identifiers that appear increases, the search time in the semantic tagging service will also increase, thereby affecting performance; cyclomatic complexity. As a measure of code complexity, an increase in cyclomatic complexity may lead to a symbol table. The depth increases, thereby affecting the performance of the "build" and "search" components; tree-related features include the number of nodes (NODE), degree (DEGREE) and depth (DEPTH). The experimental research objects of the present invention include abstract syntax trees and semantic trees. The data represented by NODE, DEGREE and DEPTH come from AST rather than the semantic tree. This is because although the implementations of AST and semantic trees are different, the semantic tree is derived from AST, suggesting a positive correlation between the size of the semantic tree and AST. In addition, in order to make the performance comparison between the two implementation methods comparable, their code feature values in the same code file should be the same. Therefore, the present invention uses AST data instead of semantic tree data. The selection of these features is also reasonable. For example, in an AST-based implementation, the "positioning" component needs to find the corresponding node in the AST according to the cursor position (row, column), and the time required may be related to the degree and depth of the AST. The "traversal" component needs to traverse all nodes in the AST, and its performance may be related to the number of nodes in the tree.
[0054] For example, prepare multiple code snippets with different cyclomatic complexities, which cover common definitions, assignments, conditional controls, loops, and function call statements in DSL. Then, perform feature statistics on these prepared code snippets, such as the number of code lines and the number of identifiers, which are recorded as δ n Then, choose a suitable feature δ iAs a control variable to generate code files, for example, the number of code lines is selected as a variable, and a group of 25 code files is generated. The minimum number of code lines of these code files is 400, and the maximum number of code lines can reach 10000. The generation step length is 400 obtained by the minimum and maximum number of code lines and the number of code files in the code file set. A group of code files is generated according to the generation step length, and the number of code lines between each group of adjacent code files differs by 400.
[0055] S4: Based on the code file set and the first execution time, a permutation importance algorithm is used to determine the key influencing features corresponding to each semantic atomic component, including: S41: Constructing a first data sample set based on the code file set, each sample in the first data sample set includes a semantic atomic component corresponding to a code file, a feature value of each code feature and an execution time of each semantic atomic component, and dividing the first data sample set into a training set and a test set according to a certain ratio; S42: Constructing a machine learning model; S43: Selecting any semantic atomic component from the unselected semantic atomic components, taking the feature value of each code feature of each code file in the training set as an input data, and taking the execution time of the semantic atomic component in the code file as a label, to train the machine learning model; S44: After the training is completed, the feature value of each code feature of each code file in the test set is input as input data to the machine learning model, and the determination coefficient is calculated based on the output predicted execution time and the execution time of the semantic atomic component in the test set. S45: Select a code feature from the unselected code features, disrupt the original correspondence between the feature value of the code feature and the code file in each code file in the test set, and then input the test set into the machine learning model, and calculate the execution time of the semantic atomic component in the test set based on the output predicted execution time by As the importance score of the code feature; S46: Repeat step S45 until each code feature in the test set is calculated, and take the code feature corresponding to the maximum importance score as the key influencing feature of the current semantic atomic component; S47: Repeat S43-S46 until all semantic atomic components obtain the corresponding key influencing features.
[0056] It can be understood that randomly selecting a code feature from the unselected code features and disrupting the original correspondence between the feature value of the code feature in each code file in the test set and the code file means: taking the selected code feature as a variable, keeping the feature values of other code features in each code file unchanged, and randomly disrupting the correspondence between the feature value of the variable in each code file and the code file, for example, taking the feature value of the variable in the first file as the feature value of the variable in the third file, and taking the feature value of the variable in the third file as the feature value of the variable in the last file.
[0057] Specifically, for the coefficient of determination The calculation is as follows:
[0058] R 2 =1-SSE / SST,
[0059] Among them, SSE is the residual sum of squares, which represents the sum of squares of the difference between the model predicted value and the actual value, and SST is the total sum of squares, which represents the sum of squares of the difference between the actual value and the mean of the true value.
[0060] Although there may be many code features in traditional software engineering, they are different from the features that may affect semantic services. After ranking importance algorithm, the key influencing features are finally determined to be code lines, number of nodes, number of defined identifiers, and number of occurring identifiers.
[0061] S5: Establish a test file set for each key influencing feature based on the code snippet, obtain the second execution time of each test file set based on the semantic service performance testing system, the second execution time includes the execution time of each semantic atomic component, and obtain the performance prediction model of each semantic atomic component based on each test file set and its corresponding second execution time.
[0062] The method of establishing a test file set based on the code snippet and the key influencing feature comprises: S51: selecting one of the key influencing features, setting the minimum value and the maximum value of the feature in the test file set, and setting the number of files in the test file set, calculating a generation step based on the number of files, the minimum value and the maximum value, based on the minimum value and the generation step, taking the key influencing feature as a variable, and using the code snippet to generate a group of test files that vary according to the generation step, thereby forming a test file set; S52: counting the minimum value and the maximum value of the feature values of other key influencing features of the test file set; S53: selecting a feature from the key influencing features that have not been selected, and calculating a generation step based on the number of files, the minimum value and the maximum value of the feature, based on the minimum value and the generation step of the feature, taking the feature as a variable, and using the code snippet to generate a group of test files that vary according to the generation step, thereby forming a test file set; S54: repeating S53 until corresponding test file sets are established for all key influencing features.
[0063] The second execution time of each test file set is obtained based on the semantic service performance test system, including: for any semantic atomic component stored on the server, each test file is sent to the server, and the service time taken by each test file to execute the semantic atomic component on the server is obtained as the second execution time; or, for any test file, the file is sent to the server, and the time taken by the file to execute each semantic atomic component on the server is obtained as the second execution time. In the above manner, the service time of each test file executing each semantic atomic component is obtained.
[0064] The performance prediction model of each semantic atomic component is obtained based on the test file sets and their corresponding second execution time, including: constructing a performance prediction model for each semantic atomic component, and obtaining a performance prediction model for each semantic atomic component by the following method: obtaining the characteristic value of the key influencing feature corresponding to the semantic atomic component in each test file based on the test files in the test file set of the semantic atomic component, and obtaining the execution time of the semantic atomic component corresponding to each test file in the second execution time; fitting the characteristic value and execution time corresponding to each code file to obtain the fitting coefficient of the performance prediction model, thereby obtaining the performance prediction model of the semantic atomic component.
[0065] For example, see Figure 2 , the corresponding relationship between the feature value of the key influencing feature of the semantic atomic component and the execution time is given, taking the abstract syntax tree and the semantic tree as the research objects, for any semantic atomic component, the feature value of the key influencing feature is used as the horizontal axis, and the execution time corresponding to the semantic atomic component is used as the vertical axis, and the observation is made. Figure 2It can be concluded that there is a linear positive correlation between the performance of the calling component and the code features. Therefore, the linear regression method is used to analyze the performance of a semantic component and the main influencing feature of the component δ i The linear relationship between them, the performance prediction formula for a single call component is:
[0066] SSC i (δ k )=k*δ k +c,
[0067] Among them, SSC i (δ k ) is the key influencing feature δ k The corresponding predicted performance of the ith semantic atomic component, k and c are fitting coefficients. For each semantic service component, the above-mentioned performance prediction model can be established. The difference is that the fitting coefficients in the performance prediction models of each component are different.
[0068] Preferably, the Abstract Syntax Tree (AST) and the Semantic Tree (STI) are obtained by converting a set of test files constructed based on DSL code snippets.
[0069] On the one hand, the performance of calling a single semantic service component (SSC) is independent of the type of semantic service, but depends on the implementation method and code size. On the other hand, the number of times a component is called is independent of the implementation method and code size, but only depends on the type of semantic service. The performance of a semantic service is equal to the sum of the performance of all its components, where the performance of a single component is equal to the performance of calling the component once multiplied by the number of times the component is called. Therefore, the performance prediction formula of the semantic service is:
[0070]
[0071] Among them, SSP is semantic service performance, SSC i is the prediction performance of the i-th semantic atomic component, times i is the execution times of the i-th semantic atomic component, i=1, 2, 3, ..., n.
[0072] S6: Obtain the semantic service to be tested, obtain the semantic atomic components corresponding to the semantic service to be tested and their execution times based on the semantic service database query, and obtain the performance prediction result of the semantic service based on the performance prediction model of each semantic atomic component and the execution times.
[0073] For example, for a renaming service, the service is expected to call a table lookup component and a traversal component, where the main influencing feature of the table lookup component is the number of defined identifiers δDEF , the main influencing feature of the traversal component is the number of lines of code δ LOC , and its semantic service performance SSP prediction formula is:
[0074] SSP Rename =SSC search (δ DEF )+SSC taverse (δ LOC );
[0075] For the code highlighting service, the service is expected to call the traversal component once and the table lookup component the number of times it calls is equal to the number of occurrence identifiers (OCC) in the file. The performance prediction formula is:
[0076] SSP Semantic Tokens =SSC search (δ DEF ) OCC+SSC traverse (δ LOC ).
[0077] Compared with the prior art, the present embodiment provides a semantic service performance prediction method based on component and scale characteristics. By constructing a semantic service database and a semantic service performance testing system, a permutation importance algorithm is used to determine the key influencing features corresponding to each semantic atomic component based on a code file set and a first execution time, a test file set of each key influencing feature is established based on a code fragment, and a performance prediction model of each semantic atomic component is obtained based on each test file set and its corresponding second execution time. This realizes rapid prediction of semantic service performance and reduces the time cost and physical resource consumption of semantic service performance prediction. The established performance prediction model has wider adaptability, higher accuracy, and stronger versatility.
[0078] Those skilled in the art will appreciate that all or part of the processes of the above-described embodiment methods may be implemented by instructing related hardware through a computer program, and the program may be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.
[0079] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.
Claims
1. A semantic service performance prediction method based on component and scale characteristics, characterized in that: include: S1: constructing a semantic service database, wherein the semantic service database stores various semantic services and semantic atomic components and their call times contained in various semantic services; S2: constructing a semantic service performance test system, wherein the semantic service performance test system is used to obtain the execution time of the semantic service and the semantic atomic component; S3: construct a code snippet, generate a code file set based on the code snippet, and use the semantic service performance testing system to obtain a first execution time of the code file set, where the first execution time includes the execution time of each semantic atomic component; S4: Determine key impact features corresponding to each semantic atomic component using a permutation importance algorithm based on the code file set and the first execution time; S5: establishing a test file set of each key influencing feature based on the code snippet, obtaining a second execution time of each test file set based on the semantic service performance testing system, wherein the second execution time includes the execution time of each semantic atomic component, and obtaining a performance prediction model of each semantic atomic component based on each test file set and its corresponding second execution time; S6: Obtain the semantic service to be tested, obtain the semantic atomic components corresponding to the semantic service to be tested and their execution times based on the semantic service database query, and obtain the performance prediction result of the semantic service based on the performance prediction model of each semantic atomic component and the execution times.
2. According to claim 1, a semantic service performance prediction method based on component and scale characteristics is characterized in that: The semantic service performance testing system is implemented based on vscode / test-web, and includes a server and a client. The server is used to write and store semantic services, and the client is used to request semantic services and fill in parameters related to the semantic services.
3. A semantic service performance prediction method based on component and scale characteristics according to claim 2, characterized in that: The semantic services include: semantic tokens, document highlighting, file symbols, naming, workspace symbols, go to declaration, go to definition, go to type definition, go to implementation, find reference, call hierarchy, type hierarchy, document link, hover, inline value inlay prompt, signature help, scope filter completion, symbol type filter completion, function signature completion, existence check, type consistency check, accessibility check; the semantic atomic components include: construction, positioning, search, traversal; the number of calls is the number of executions of each semantic atomic component in the semantic service.
4. A method for predicting semantic service performance based on component and scale characteristics according to claim 3, characterized in that: The constructing code snippets and generating a code file set based on the code snippets comprises: S31: Construct DSL code snippets with different cyclomatic complexities; S32: Select any code feature as a variable, set the minimum value, maximum value and number of code files in the code file set of the code feature, calculate the generation step based on the number of code files, the minimum value and maximum value of the code feature, based on the minimum value and the generation step, use the code feature as a variable, use the code fragment to generate a group of code files that vary according to the generation step, and thus obtain a code file set.
5. A method for predicting semantic service performance based on component and scale characteristics according to claim 4, characterized in that: Determining key impact features corresponding to each semantic atomic component by using a permutation importance algorithm based on the code file set and the first execution time includes: S41: constructing a first data sample set based on the code file set, wherein each sample in the first data sample set includes a semantic atomic component corresponding to a code file, a feature value of each code feature, and an execution time of each semantic atomic component, and dividing the first data sample set into a training set and a test set according to a certain ratio; S42: Build machine learning models; S43: selecting any semantic atomic component from the unselected semantic atomic components, taking the feature value of each code feature of each code file in the training set as an input data, and taking the execution time of the semantic atomic component in the code file as a label, to train the machine learning model; S44: After the training is completed, the feature values of each code feature of each code file in the test set are input into the machine learning model as input data, and the determination coefficient is calculated based on the output predicted execution time and the execution time of the semantic atomic component in the test set. S45: Select a code feature from the unselected code features, disrupt the original correspondence between the feature value of the code feature and the code file in each code file in the test set, and then input the test set into the machine learning model, and calculate the execution time of the semantic atomic component in the test set based on the output predicted execution time by As the importance score of the code feature; S46: repeat step S45 until each code feature in the test set is calculated, and the code feature corresponding to the maximum importance score is used as the key influencing feature of the current semantic atomic component; S47: Repeat S43-S46 until all semantic atomic components obtain corresponding key impact features.
6. A method for predicting semantic service performance based on component and scale characteristics according to claim 5, characterized in that: The code features include: number of code lines, number of defined identifiers, number of occurring identifiers, cyclomatic complexity, number of nodes, degree, and depth.
7. A method for predicting semantic service performance based on component and scale characteristics according to claim 6, characterized in that: The establishing of a test file set based on the code snippet and the key impact feature comprises: S51: selecting a feature in the key influencing features, setting the minimum value and maximum value of the feature in the test file set, and setting the number of files in the test file set, calculating a generation step length based on the number of files, the minimum value and the maximum value, and based on the minimum value and the generation step length, using the key influencing feature as a variable, using the code snippet to generate a group of test files that vary according to the generation step length, thereby forming a test file set; S52: Counting the minimum and maximum values of other key influencing feature values of the test file set; S53: selecting a feature from the unselected key influencing features, and calculating a generation step length based on the number of files, the minimum value and the maximum value of the feature, and based on the minimum value and the generation step length of the feature, using the feature as a variable, and using the code snippet to generate a set of test files that vary according to the generation step length, thereby forming a test file set; S54: Repeat S53 until corresponding test file sets are established for all key influencing features.
8. The method for predicting semantic service performance based on component and scale characteristics according to claim 7, characterized in that: The obtaining of the performance prediction model of each semantic atomic component based on each test file set and its corresponding second execution time includes: Construct a performance prediction model for each semantic atomic component; obtain the performance prediction model for each semantic atomic component through the following method: Based on the test files in the test file set of the semantic atomic component, the characteristic values of the key influencing features corresponding to the semantic atomic component in each test file are obtained, and the execution time of the semantic atomic component corresponding to each test file in the second execution time is obtained; the characteristic values and execution times corresponding to each code file are fitted to obtain the fitting coefficients of the performance prediction model, thereby obtaining the performance prediction model of the semantic atomic component.
9. A method for predicting semantic service performance based on component and scale characteristics according to claim 8, characterized in that: The performance prediction model of the constructed semantic atomic component is: SSC i (d k )=k*δ k +c, Among them, SSC i (δ k ) is the key influencing feature δ k The corresponding prediction performance of the i-th semantic atomic component, k and c are the fitting coefficients.
10. A method for predicting semantic service performance based on component and scale characteristics according to claim 9, characterized in that: The performance prediction results of the semantic service are obtained through the following formula: Among them, SSP is semantic service performance, SSC i is the prediction performance of the i-th semantic atomic component, times i is the execution times of the i-th semantic atomic component, i=1, 2, 3, ..., n.
