Comparative Evaluation Methods and Devices for Programming Languages

Abstract

This application relates to a comparative evaluation method and apparatus for programming languages, comprising: acquiring multiple target programming languages ​​to be evaluated, wherein the target programming languages ​​include at least a first type of programming language and a second type of programming language; establishing a corresponding independent running environment for each target programming language; distributing a preset general test case set to each of the independent running environments for execution testing, and collecting raw test data generated by each target programming language during the execution of the general test case set; normalizing each of the raw test data according to a preset standardized mapping rule to construct a standard feature vector corresponding to each target programming language; and calculating the spatial distance of each standard feature vector based on a preset feature space difference measurement model to obtain the evaluation results of the multiple target programming languages.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a comparative evaluation method and apparatus for programming languages. Background Technology

[0002] As the software industry continues to evolve, the programming language ecosystem presents a pattern of "mainstream languages ​​dominating for a long time, while emerging languages ​​are constantly emerging." However, current mainstream international programming language testing and benchmarking tools (such as SPEC and BenchmarkGame) are designed with underlying parsing engines and runtime probing logic tailored to traditional mainstream programming languages ​​such as C, Java, and Python, resulting in significant syntax binding and runtime coupling issues. Specifically, existing testing tools cannot effectively adapt to the unique syntax structures, memory management mechanisms, and runtime environments of emerging programming languages, leading to dilemmas such as "no tools available" or "forced adaptation resulting in distorted results" when testing emerging programming languages.

[0003] This technological incompatibility directly leads to a lack of unified quantitative evaluation standards for emerging programming languages, making it difficult to conduct objective and fair horizontal comparisons with traditional mainstream programming languages ​​in key dimensions such as functional completeness, execution performance, and security. During the technology promotion and application selection phase, the lack of authoritative third-party testing data makes emerging programming languages ​​highly susceptible to developer skepticism, severely hindering their ecosystem development and market penetration. Summary of the Invention

[0004] This application provides a comparative evaluation method and apparatus for programming languages ​​to achieve an equivalent evaluation of emerging programming languages ​​and mainstream international programming languages.

[0005] Firstly, this application provides a comparative evaluation method for programming languages, including: A variety of target programming languages ​​to be evaluated are obtained, including at least a first type of programming language and a second type of programming language. For each target programming language, a corresponding independent runtime environment is established. A pre-defined general test case set is distributed to each independent runtime environment for execution testing, and raw test data generated by each target programming language during the execution of the general test case set is collected. The raw test data is normalized according to a pre-defined standardized mapping rule to construct a standard feature vector corresponding to each target programming language. Based on a pre-defined feature space difference measurement model, spatial distance is calculated for each standard feature vector to obtain the evaluation results for the various target programming languages.

[0006] In one possible implementation, the method further includes: Extract common low-level instruction abstraction structures from the first type of programming language and the second type of programming language; based on the low-level instruction abstraction structures, and combined with preset abnormal attack injection test vectors, construct a cross-language and compatible general test case set.

[0007] In one possible implementation, the step of normalizing each of the original test data according to a preset standardized mapping rule to construct a standard feature vector corresponding to each of the target programming languages ​​includes: Resource consumption indicators, function execution status, and security defense response characteristics are extracted from each of the original test data. Using a preset dimension conversion mapping table, the resource consumption indicators, function execution status, and security defense response characteristics are converted into corresponding numerical sequences with unified dimensions. Multiple numerical sequences are combined and spliced ​​to construct a standard feature vector corresponding to each of the target programming languages.

[0008] In one possible implementation, the step of using a preset dimension conversion mapping table to convert the resource consumption index, function execution status, and security defense response characteristics into corresponding numerical sequences with unified dimensions includes: Obtain the execution code logic complexity parameters corresponding to each of the original test data; determine the normalized penalty weight of the resource consumption index based on the execution code logic complexity parameters; convert the resource consumption index based on the dimension conversion mapping table carrying the normalized penalty weight to obtain a first target dimension value; use the dimension conversion mapping table to convert the function execution status and security defense response characteristics to obtain a second target dimension value; fuse the first target dimension value and the second target dimension value to obtain a corresponding unified dimension value sequence.

[0009] In one possible implementation, the spatial distance calculation is performed on each of the standard feature vectors based on a preset feature space difference measurement model to obtain evaluation results for multiple target programming languages, including: In a multidimensional feature space, a first standard feature vector corresponding to the first type of programming language and a second standard feature vector corresponding to the second type of programming language are determined; the feature aggregation distance between the first standard feature vector and the second standard feature vector is calculated; based on the feature aggregation distance, the performance difference between the first type of programming language and the second type of programming language is quantitatively evaluated, and the evaluation result of the target programming language is generated based on the performance difference.

[0010] In one possible implementation, the step of quantifying and evaluating the performance difference between the first type of programming language and the second type of programming language based on the feature aggregation distance, and generating an evaluation result for the target programming language based on the performance difference, includes: Obtain the total number of test tasks contained in the general test case set; determine a baseline distance threshold based on the total number of test tasks; compare the feature aggregation distance with the baseline distance threshold, and determine the quantization score between the first type of programming language and the second type of programming language based on the comparison result; add the quantization score as a performance difference to the evaluation result of the target programming language.

[0011] In one possible implementation, the method further includes: Extract target feature dimensions indicating performance weaknesses from the evaluation results; match language-level optimization suggestions based on the target feature dimensions from a preset historical optimization rule base; add the language-level optimization suggestions to the evaluation results of the target programming language.

[0012] Secondly, this application provides a comparative evaluation device for programming languages, comprising: An acquisition module is used to acquire multiple target programming languages ​​to be evaluated, wherein the target programming languages ​​include at least a first type of programming language and a second type of programming language; A module is established to create a corresponding independent runtime environment for each of the target programming languages. The testing module is used to distribute a preset set of general test cases to each of the independent runtime environments for execution testing, and to collect raw test data generated by each of the target programming languages ​​during the execution of the set of general test cases. The construction module is used to normalize each of the original test data according to a preset standardized mapping rule, and construct a standard feature vector corresponding to each of the target programming languages. The processing module is used to calculate the spatial distance of each of the standard feature vectors based on a preset feature space difference measurement model, so as to obtain the evaluation results of multiple target programming languages.

[0013] Thirdly, this application provides a computer device, including: a processor and a memory, wherein the processor is configured to execute a comparison and evaluation program of a programming language stored in the memory, so as to implement the comparison and evaluation method of the programming language described in any one of the first aspects above.

[0014] Fourthly, this application provides a storage medium storing one or more programs that can be executed by one or more processors to implement the comparative evaluation method of the programming languages ​​described in any of the first aspects above.

[0015] The comparative evaluation method for programming languages ​​provided in this application involves acquiring multiple target programming languages ​​to be evaluated, including at least a first type of programming language and a second type of programming language; establishing a corresponding independent runtime environment for each target programming language; distributing a preset general test case set to each independent runtime environment for execution testing, and collecting raw test data generated by each target programming language during the execution of the general test case set; normalizing each raw test data according to a preset standardized mapping rule to construct a standard feature vector corresponding to each target programming language; and calculating the spatial distance of each standard feature vector based on a preset feature space difference measurement model to obtain the evaluation results of multiple target programming languages. This method can achieve the following technical effects: This invention organically integrates various technologies such as automated testing, static analysis, and dynamic monitoring to form a comprehensive detection capability. Addressing hidden defects that existing testing methods struggle to cover, this invention can comprehensively and efficiently uncover potential problems in different programming languages ​​during runtime, accurately identifying performance bottlenecks such as excessive memory usage and low execution efficiency. Compared to traditional manual testing or single-tool testing, it not only significantly improves the accuracy of defect detection and code coverage but also effectively shortens the regression testing cycle and reduces decision-making costs caused by information asymmetry.

[0016] It can generate structured full test reports and fine-grained error diagnosis information, overcoming the limitations of existing technologies that only provide a single pass / fail result. Users can quickly grasp the syntax features, runtime performance, and stability boundaries of the target programming language, thereby making more accurate judgments in language selection, code refactoring, and other stages, improving development efficiency from the source, and ensuring the overall quality of delivered software. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0019] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0020] Figure 1 A flowchart illustrating a comparative evaluation method for programming languages ​​provided in this application embodiment; Figure 2 A flowchart illustrating another comparative evaluation method for programming languages ​​provided in this application embodiment; Figure 3 A flowchart illustrating the process of determining a numerical sequence with uniform dimensions in another comparative evaluation method for programming languages ​​provided in this application embodiment; Figure 4 This application provides a schematic diagram of the structure of a comparative evaluation device for programming languages; Figure 5 This application provides a schematic diagram of the structure of a computer device. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] The following disclosure provides numerous different embodiments or examples for implementing various structures of the invention. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of the invention. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0023] Figure 1 A flowchart illustrating a comparative evaluation method for programming languages ​​provided in this application embodiment is shown below. Figure 1 The method specifically includes: S11. Obtain multiple target programming languages ​​to be evaluated, wherein the target programming languages ​​include at least a first type of programming language and a second type of programming language.

[0024] In this embodiment, to achieve a comprehensive and objective evaluation of different programming languages, it is first necessary to acquire multiple target programming languages ​​to be evaluated. These target programming languages ​​include at least a first category and a second category. The first category can be understood as emerging programming languages ​​(e.g., Rust, Cangjie, etc.), and the second category can be understood as mainstream international programming languages ​​(e.g., C, C++, Java, Python, etc.). The system can automatically acquire multiple target programming languages ​​for horizontal comparison by receiving configuration selection commands from the user on the front-end interface or by using a pre-set list of periodic evaluation tasks. This provides a clear evaluation target for subsequent multi-dimensional testing and quantitative evaluation.

[0025] S12. For each of the target programming languages, establish a corresponding independent runtime environment.

[0026] In this embodiment of the application, in order to avoid interference between the runtime environments of different languages ​​and to ensure the objectivity and accuracy of the test results, a corresponding independent runtime environment is established for each target programming language.

[0027] Furthermore, by invoking the container management module, isolated runtime containers are created for each target programming language, and pre-configured images matching that language (such as GCC, JDK, Cangjie VM, and other language compiler or interpreter images) are automatically pulled, completing dependency management and runtime environment initialization. In this way, it is ensured that all target programming languages ​​execute in a consistent and independent underlying platform environment, eliminating the interference of environmental differences on test data.

[0028] S13. Distribute the preset general test case set to each of the independent operating environments for execution testing, and collect the raw test data generated by each of the target programming languages ​​during the execution of the general test case set.

[0029] In this embodiment of the application, the preset general test case set (covering multiple dimensions such as basic syntax, concurrency mechanism, and memory management) is distributed to each independent running environment for batch execution testing through the automated test scheduling module.

[0030] During test execution, the raw test data generated by each target programming language in the process of executing the general test case set is monitored and collected in real time. This raw test data includes, but is not limited to: compilation time, runtime, CPU load, memory usage, number of system calls, exception stack and security event logs, etc., and all indicators are reported to the central control module for structured storage.

[0031] S14. Normalize each of the original test data according to the preset standardized mapping rules to construct a standard feature vector corresponding to each of the target programming languages.

[0032] In this embodiment of the application, since the test data output by different programming languages ​​differ in measurement units and description specifications, it is necessary to standardize the collected data; according to the preset standardization mapping rules, the original test data are normalized by unifying the output coding specifications, exception description templates and resource consumption measurement units.

[0033] For example, the time units of different language outputs are uniformly converted to milliseconds, the memory units are uniformly converted to megabytes, and standardized feature vectors are generated based on multi-dimensional features (such as average execution time, maximum memory usage, function pass rate, and number of security vulnerabilities) to construct standard feature vectors corresponding to each target programming language, thus forming a structured language test dataset.

[0034] S15. Based on the preset feature space difference measurement model, spatial distance is calculated for each of the standard feature vectors to obtain evaluation results for multiple target programming languages.

[0035] In this embodiment, based on a preset feature space difference measurement model (such as using cosine similarity, Euclidean distance, or K-means clustering), each standard feature vector is mapped to a multi-dimensional feature space, and spatial distance is calculated for each standard feature vector. By calculating the distance relationship and aggregation degree between emerging and mainstream languages ​​under different testing dimensions, performance differences, functional completeness, security robustness, and other indicators are quantified. Finally, evaluation results for multiple target programming languages, including comprehensive ranking and subdivided dimension scores, are automatically generated.

[0036] Figure 2 A flowchart illustrating another comparative evaluation method for programming languages ​​provided in this application embodiment is shown below. Figure 2 The method specifically includes: S21. Obtain multiple target programming languages ​​to be evaluated, wherein the target programming languages ​​include at least a first type of programming language and a second type of programming language.

[0037] S22. For each of the target programming languages, establish a corresponding independent runtime environment.

[0038] In this embodiment, S21-S22 and Figure 1 Similar to S11-S12, please refer to Figure 1 The relevant description is concise and will not be elaborated upon here.

[0039] S23. Extract the common low-level instruction abstraction structure from the first type of programming language and the second type of programming language.

[0040] In this embodiment of the application, in order to find commonalities among different programming languages, the syntax definitions and operation mechanisms of the first type of programming language and the second type of programming language are analyzed, and the underlying instruction abstract structure that is common to both is extracted. The underlying instruction abstract structure includes, but is not limited to, abstract expressions such as basic data type definitions, logical control flow (such as loops and branches), function call specifications and memory allocation mechanisms, thereby providing unified underlying logic support for the generation of cross-language use cases.

[0041] S24. Based on the underlying instruction abstraction structure, and combined with the preset abnormal attack injection test vector, construct a cross-language and compatible general test case set.

[0042] In this embodiment, after extracting the common underlying instruction abstraction structure, test cases are constructed to further meet security testing requirements. Based on the underlying instruction abstraction structure, and combined with preset abnormal attack injection test vectors (such as test code snippets simulating SQL injection, buffer overflow, null pointer reference, and other security threats), test modules containing functional verification, performance benchmarking, and security operations are generated. These modules are organized using a semantic tagging system, ultimately constructing a cross-language and compatible universal test case set, ensuring that the same set of test cases can be correctly parsed and executed in independent runtime environments of different languages.

[0043] S25. Distribute the preset general test case set to each of the independent operating environments for execution testing, and collect the raw test data generated by each of the target programming languages ​​during the execution of the general test case set.

[0044] In this embodiment, a distributed task scheduling mechanism distributes the constructed general test case set to each independent runtime environment for execution testing. The system runs all test cases in batches according to the task execution plan and collects raw test data for each target programming language in real time during execution, including output results, execution time, memory usage, thread behavior, and exception stacks. Each test task execution log is automatically archived and labeled with metadata such as language version and test case identifier for subsequent data analysis.

[0045] S26. Extract resource consumption indicators, function execution status, and security defense response characteristics from each of the original test data.

[0046] In this embodiment of the application, after collecting massive amounts of raw test data, it is necessary to extract features from it in multiple dimensions; extract resource consumption indicators that reflect system overhead (such as peak CPU load, average memory usage, running time, etc.) from each raw test data, extract functional execution status that reflects code correctness (such as test case pass rate, number of assertion successes, number of compilation errors, etc.), and extract security defense response features that reflect the language's ability to deal with security threats (such as number of vulnerability interceptions, success rate of boundary anomaly handling, and unauthorized access blocking rate, etc.), thereby comprehensively characterizing the running performance of the programming language.

[0047] S27. Using a preset dimension conversion mapping table, the resource consumption index, function execution status, and security defense response characteristics are converted into corresponding numerical sequences with unified dimensions.

[0048] In this embodiment of the application, since the extracted features have huge differences in physical meaning and numerical range, it is necessary to unify their dimensions. Using a preset dimension conversion mapping table, resource consumption indicators with different physical units (such as milliseconds, megabytes), function execution status representing status (such as percentage, Boolean value), and security defense response features representing the number of times are mapped and converted into corresponding numerical sequences of the same dimension (such as 0 to 100 points) within the same numerical range, thereby eliminating the interference of dimension differences on subsequent spatial distance calculations.

[0049] Figure 3 A flowchart illustrating the determination of a numerical sequence with uniform dimensions in another comparative evaluation method for programming languages ​​provided in this application embodiment is shown below. Figure 3 The method specifically includes: S271. Obtain the execution code logic complexity parameters corresponding to each of the original test data.

[0050] Before converting the resource consumption metrics to their dimensions, to consider the impact of the complexity of the test cases on the reasonableness of resource consumption, static code analysis tools are used to obtain the execution code logic complexity parameters corresponding to each original test data. These parameters can be used to quantify the logical complexity of the current test case through metrics such as the number of lines of code, nesting depth, loop complexity, or algorithm time complexity.

[0051] S272. Determine the normalized penalty weight of the resource consumption index based on the execution code logic complexity parameter.

[0052] Furthermore, the evaluation criteria for resource consumption are dynamically adjusted; the normalized penalty weight of the resource consumption index is determined based on the logic complexity parameter of the executed code; for example, when the code logic complexity is high, it is allowed to occupy more computing resources, and a smaller normalized penalty weight is assigned; conversely, when the code logic is simple but the resource consumption is huge, a larger normalized penalty weight is assigned, thereby ensuring the accuracy of resource consumption assessment.

[0053] S273. Based on the dimension conversion mapping table carrying the normalization penalty weight, the resource consumption index is converted to obtain the first target dimension value.

[0054] Furthermore, the resource consumption indicators are specifically transformed; based on the dimension transformation mapping table carrying normalized penalty weights, the original resource consumption indicators (such as actual memory usage) are substituted into the mapping function, and weighted calculation or nonlinear mapping is performed in combination with the penalty weights to transform the physical dimension resource consumption indicators into dimensionless first target dimension values ​​(such as resource consumption scores). The higher the value, the better the resource utilization efficiency.

[0055] S274. The dimension conversion mapping table is used to convert the function execution status and security defense response characteristics to obtain the second target dimension value.

[0056] Furthermore, for features in the functional and security dimensions, conventional mapping methods are used for conversion; using a dimensional conversion mapping table, the functional execution status (such as test pass rate) and security defense response features (such as vulnerability interception success rate) are directly converted according to preset linear or step mapping rules to obtain the corresponding second target dimensional values ​​(such as functional score and security score) to uniformly reflect the language's performance in functional completeness and security robustness.

[0057] S275. The first target dimension value and the second target dimension value are fused to obtain a corresponding unified dimension value sequence.

[0058] Finally, the numerical values ​​are integrated; the first target dimension value representing resource consumption and the second target dimension value representing functional and security performance are spliced ​​or weighted and fused according to a preset dimensional order to obtain a corresponding numerical sequence with unified dimensions. This sequence reflects the comprehensive performance of the target programming language in a single or single type of test task.

[0059] S28. Combine and concatenate multiple numerical sequences to construct a standard feature vector corresponding to each target programming language.

[0060] In this embodiment, vector construction is performed by combining and concatenating multiple numerical sequences obtained from functional testing, performance evaluation, security verification, and other subdivisions of the same target programming language to form a high-dimensional multidimensional array. This constructs a standard feature vector corresponding to each target programming language, providing standardized input data for subsequent model calculations.

[0061] S29. In the multidimensional feature space, determine the first standard feature vector corresponding to the first type of programming language and the second standard feature vector corresponding to the second type of programming language.

[0062] Furthermore, it is mapped to a high-dimensional space for classification; in the multi-dimensional feature space, based on the language type label, the standard feature vector corresponding to the first category of programming languages ​​belonging to emerging programming languages ​​is determined as the first standard feature vector, and the standard feature vector corresponding to the second category of programming languages ​​belonging to the international mainstream programming languages ​​is determined as the second standard feature vector, thereby clarifying the distribution position of the two types of languages ​​at the spatial level.

[0063] S210. Calculate the feature aggregation distance between the first standard feature vector and the second standard feature vector.

[0064] In this embodiment, to quantify the overall differences between the two types of languages ​​and perform spatial distance calculations, the platform employs distance measurement algorithms such as Euclidean distance, Manhattan distance, or cosine similarity to calculate the feature aggregation distance between the first standard feature vector (or the cluster center vector of the first type of language) and the second standard feature vector (or the cluster center vector of the second type of language). This feature aggregation distance intuitively reflects the overall deviation of the emerging programming language from mainstream programming languages ​​in terms of comprehensive functionality, performance, and security.

[0065] S211. Based on the feature aggregation distance, quantitatively evaluate the performance difference between the first type of programming language and the second type of programming language, and generate the evaluation result of the target programming language based on the performance difference.

[0066] In this embodiment of the application, based on feature aggregation distance, the performance difference between the first type of programming language and the second type of programming language (such as difference percentage or benchmark score) is quantitatively evaluated through a preset distance-score mapping function. Based on this performance difference, combined with the specific scores of each subdivision dimension, an evaluation result of the target programming language is generated, which includes comprehensive ranking, radar chart data and advantages and disadvantages analysis, to assist users in making selections.

[0067] In one optional embodiment of this application, the total number of test tasks included in the general test case set is obtained; a baseline distance threshold is determined based on the total number of test tasks; the feature aggregation distance is compared with the baseline distance threshold, and a quantization score between the first type of programming language and the second type of programming language is determined based on the comparison result; the quantization score is added as a performance difference to the evaluation result of the target programming language.

[0068] Furthermore, to make the evaluation results more objective and adaptable to test scenarios of different scales, a dynamic threshold mechanism is introduced. First, the total number of test tasks included in the general test case set (i.e., the total number of test cases participating in the evaluation) is obtained. Since the more test tasks there are, the higher the dimension of the feature vector, and the greater the absolute value of the spatial distance, the platform dynamically determines the baseline distance threshold based on the total number of test tasks. Then, the calculated feature aggregation distance is compared with the baseline distance threshold (e.g., the ratio is calculated), and the quantitative score between the first type of programming language and the second type of programming language (e.g., the relative score under a full score of 100) is determined based on the comparison result. This quantitative score is added to the evaluation result as the performance difference, thereby avoiding scoring distortion caused by changes in the number of test cases.

[0069] S212. Extract target feature dimensions that indicate performance weaknesses from the evaluation results.

[0070] In this embodiment of the application, the scores of each dimension in the evaluation results are analyzed to identify the feature dimensions with scores lower than the preset standard line or the largest gap with the mainstream language (such as "high-concurrency processing takes too long" or "low memory leak interception rate"). These target feature dimensions that indicate weak performance are extracted and used as trigger conditions for generating optimization suggestions in the future.

[0071] S213. Match language underlying optimization suggestion information based on the target feature dimension in the preset historical optimization rule base.

[0072] In this embodiment, a knowledge base is used to generate targeted suggestions. A preset historical optimization rule base (which stores expert knowledge bases and historical optimization cases) is accessed, and rule matching and similar case retrieval algorithms are used to find language-level optimization suggestions corresponding to the target feature dimension. For example, if the target feature dimension is "low memory management efficiency", suggestions for compiler or syntax-level improvements such as "optimize garbage collection mechanism" or "introduce smart pointer features" are matched.

[0073] S214. Add the language underlying optimization suggestion information to the evaluation result of the target programming language.

[0074] In this embodiment of the application, the matched language underlying optimization suggestions (including syntax improvement suggestions, compiler optimization directions, security mechanism enhancements, etc.) are structurally added to the evaluation results of the target programming language to generate a complete test report containing "benchmark" rankings, difference analysis, and optimization guidance.

[0075] The comparative evaluation method for programming languages ​​provided in this application involves acquiring multiple target programming languages ​​to be evaluated, including at least a first type of programming language and a second type of programming language; establishing a corresponding independent runtime environment for each target programming language; distributing a preset general test case set to each independent runtime environment for execution testing, and collecting raw test data generated by each target programming language during the execution of the general test case set; normalizing each raw test data according to a preset standardized mapping rule to construct a standard feature vector corresponding to each target programming language; and calculating the spatial distance of each standard feature vector based on a preset feature space difference measurement model to obtain the evaluation results of multiple target programming languages. This method can achieve the following technical effects: This invention organically integrates various technologies such as automated testing, static analysis, and dynamic monitoring to form a comprehensive detection capability. Addressing hidden defects that existing testing methods struggle to cover, this invention can comprehensively and efficiently uncover potential problems in different programming languages ​​during runtime, accurately identifying performance bottlenecks such as excessive memory usage and low execution efficiency. Compared to traditional manual testing or single-tool testing, it not only significantly improves the accuracy of defect detection and code coverage but also effectively shortens the regression testing cycle and reduces decision-making costs caused by information asymmetry.

[0076] It can generate structured full test reports and fine-grained error diagnosis information, overcoming the limitations of existing technologies that only provide a single pass / fail result. Users can quickly grasp the syntax features, runtime performance, and stability boundaries of the target programming language, thereby making more accurate judgments in language selection, code refactoring, and other stages, improving development efficiency from the source, and ensuring the overall quality of delivered software.

[0077] Figure 4 This application provides a schematic diagram of the structure of a programming language comparison and evaluation device, such as... Figure 4 The device specifically includes: The acquisition module 401 is used to acquire multiple target programming languages ​​to be evaluated, wherein the target programming languages ​​include at least a first type of programming language and a second type of programming language; Module 402 is used to establish a corresponding independent runtime environment for each of the target programming languages. Test module 403 is used to distribute a preset set of general test cases to each of the independent running environments for execution testing, and to collect raw test data generated by each of the target programming languages ​​during the execution of the set of general test cases; The construction module 404 is used to normalize each of the original test data according to a preset standardized mapping rule, and construct a standard feature vector corresponding to each of the target programming languages. The processing module 405 is used to perform spatial distance calculation on each of the standard feature vectors based on a preset feature space difference measurement model, so as to obtain the evaluation results of multiple target programming languages.

[0078] Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Figure 5 The computer device 500 shown includes at least one processor 501, a memory 502, at least one network interface 504, and other user interfaces 503. Various components in the mobile terminal 500 are coupled together via a bus system 505. It is understood that the bus system 505 is used to implement communication between these components. In addition to a data bus, the bus system 505 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 5 The general designated all buses as Bus System 505.

[0079] The user interface 503 may include a display, keyboard, or clicking device (e.g., mouse, trackball, touchpad, or touchscreen).

[0080] It is understood that the memory 502 in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 502 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0081] In some implementations, memory 502 stores elements, executable units or data structures, or subsets thereof, or extended sets thereof: operating system 5021 and application program 5022.

[0082] The operating system 5021 includes various system programs, such as the framework layer, core library layer, and driver layer, used to implement various basic business functions and handle hardware-based tasks. The application program 5022 includes various applications, such as a media player and a browser, used to implement various application functions. The program implementing the method of this embodiment can be included in the application program 5022.

[0083] In this embodiment of the invention, by calling the program or instructions stored in memory 502, specifically the program or instructions stored in application program 5022, processor 501 executes the method steps provided in each method embodiment, including, for example: A variety of target programming languages ​​to be evaluated are obtained, including at least a first type of programming language and a second type of programming language. For each target programming language, a corresponding independent runtime environment is established. A pre-defined general test case set is distributed to each independent runtime environment for execution testing, and raw test data generated by each target programming language during the execution of the general test case set is collected. The raw test data is normalized according to a pre-defined standardized mapping rule to construct a standard feature vector corresponding to each target programming language. Based on a pre-defined feature space difference measurement model, spatial distance is calculated for each standard feature vector to obtain the evaluation results for the various target programming languages.

[0084] In one possible implementation, a common low-level instruction abstraction structure is extracted from the first type of programming language and the second type of programming language; based on the low-level instruction abstraction structure, combined with a preset abnormal attack injection test vector, a cross-language and compatible general test case set is constructed.

[0085] In one possible implementation, resource consumption indicators, function execution status, and security defense response characteristics are extracted from each of the original test data; using a preset dimension conversion mapping table, the resource consumption indicators, function execution status, and security defense response characteristics are converted into corresponding numerical sequences with unified dimensions; multiple numerical sequences are combined and spliced ​​to construct a standard feature vector corresponding to each of the target programming languages.

[0086] In one possible implementation, the following steps are taken: First, the execution code logic complexity parameters corresponding to each of the original test data are obtained. Second, the normalized penalty weight of the resource consumption index is determined based on the execution code logic complexity parameters. Third, the resource consumption index is converted using the dimension conversion mapping table carrying the normalized penalty weight to obtain a first target dimension value. Fourth, the function execution state and security defense response characteristics are converted using the dimension conversion mapping table to obtain a second target dimension value. Fifth, the first target dimension value and the second target dimension value are fused to obtain a corresponding unified dimension value sequence.

[0087] In one possible implementation, in a multidimensional feature space, a first standard feature vector corresponding to the first type of programming language and a second standard feature vector corresponding to the second type of programming language are determined; the feature aggregation distance between the first standard feature vector and the second standard feature vector is calculated; based on the feature aggregation distance, the performance difference between the first type of programming language and the second type of programming language is quantitatively evaluated, and an evaluation result of the target programming language is generated based on the performance difference.

[0088] In one possible implementation, the total number of test tasks contained in the general test case set is obtained; a baseline distance threshold is determined based on the total number of test tasks; the feature aggregation distance is compared with the baseline distance threshold, and a quantization score between the first type of programming language and the second type of programming language is determined based on the comparison result; the quantization score is added as a performance difference to the evaluation result of the target programming language.

[0089] In one possible implementation, target feature dimensions indicating performance weaknesses are extracted from the evaluation results; language-level optimization suggestions based on the target feature dimensions are matched in a preset historical optimization rule base; and the language-level optimization suggestions are added to the evaluation results of the target programming language.

[0090] The methods disclosed in the above embodiments of the present invention can be applied to or implemented by processor 501. Processor 501 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 501 or by instructions in the form of software. The processor 501 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software units in the decoding processor. The software units may be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 502. Processor 501 reads the information in memory 502 and, in conjunction with its hardware, completes the steps of the above method.

[0091] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or combinations thereof.

[0092] For software implementation, the techniques described herein can be implemented by units that perform the functions described herein. The software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0093] The computer device provided in this embodiment may be as follows: Figure 5 The computer device shown can perform, for example Figure 1-3 All steps of the comparative evaluation method for Chinese programming languages, thereby achieving Figure 1-3 For details on the technical effectiveness of the comparative evaluation methods for the programming languages ​​shown, please refer to [link / reference needed]. Figure 1-3 The relevant descriptions are presented concisely and will not be elaborated upon here.

[0094] This invention also provides a storage medium (computer-readable storage medium). This storage medium stores one or more programs. The storage medium may include volatile memory, such as random access memory; it may also include non-volatile memory, such as read-only memory, flash memory, hard disk, or solid-state drive; and it may also include combinations of the above types of memory.

[0095] When one or more programs in the storage medium can be executed by one or more processors to implement the programming language comparison and evaluation method described above, which is executed on the programming language comparison and evaluation device side.

[0096] The processor is used to execute a comparison and evaluation program of a programming language stored in memory to implement the following steps of a comparison and evaluation method of a programming language executed on the programming language comparison and evaluation device side: A variety of target programming languages ​​to be evaluated are obtained, including at least a first type of programming language and a second type of programming language. For each target programming language, a corresponding independent runtime environment is established. A pre-defined general test case set is distributed to each independent runtime environment for execution testing, and raw test data generated by each target programming language during the execution of the general test case set is collected. The raw test data is normalized according to a pre-defined standardized mapping rule to construct a standard feature vector corresponding to each target programming language. Based on a pre-defined feature space difference measurement model, spatial distance is calculated for each standard feature vector to obtain the evaluation results for the various target programming languages.

[0097] In one possible implementation, a common low-level instruction abstraction structure is extracted from the first type of programming language and the second type of programming language; based on the low-level instruction abstraction structure, combined with a preset abnormal attack injection test vector, a cross-language and compatible general test case set is constructed.

[0098] In one possible implementation, resource consumption indicators, function execution status, and security defense response characteristics are extracted from each of the original test data; using a preset dimension conversion mapping table, the resource consumption indicators, function execution status, and security defense response characteristics are converted into corresponding numerical sequences with unified dimensions; multiple numerical sequences are combined and spliced ​​to construct a standard feature vector corresponding to each of the target programming languages.

[0099] In one possible implementation, the following steps are taken: First, the execution code logic complexity parameters corresponding to each of the original test data are obtained. Second, the normalized penalty weight of the resource consumption index is determined based on the execution code logic complexity parameters. Third, the resource consumption index is converted using the dimension conversion mapping table carrying the normalized penalty weight to obtain a first target dimension value. Fourth, the function execution state and security defense response characteristics are converted using the dimension conversion mapping table to obtain a second target dimension value. Fifth, the first target dimension value and the second target dimension value are fused to obtain a corresponding unified dimension value sequence.

[0100] In one possible implementation, in a multidimensional feature space, a first standard feature vector corresponding to the first type of programming language and a second standard feature vector corresponding to the second type of programming language are determined; the feature aggregation distance between the first standard feature vector and the second standard feature vector is calculated; based on the feature aggregation distance, the performance difference between the first type of programming language and the second type of programming language is quantitatively evaluated, and an evaluation result of the target programming language is generated based on the performance difference.

[0101] In one possible implementation, the total number of test tasks contained in the general test case set is obtained; a baseline distance threshold is determined based on the total number of test tasks; the feature aggregation distance is compared with the baseline distance threshold, and a quantization score between the first type of programming language and the second type of programming language is determined based on the comparison result; the quantization score is added as a performance difference to the evaluation result of the target programming language.

[0102] In one possible implementation, target feature dimensions indicating performance weaknesses are extracted from the evaluation results; language-level optimization suggestions based on the target feature dimensions are matched in a preset historical optimization rule base; and the language-level optimization suggestions are added to the evaluation results of the target programming language.

[0103] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0104] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0105] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0106] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method of comparative evaluation of programming languages, characterized by, The method includes: Obtain multiple target programming languages ​​to be evaluated, wherein the target programming languages ​​include at least a first type of programming language and a second type of programming language; For each of the target programming languages, a corresponding independent runtime environment shall be established; A pre-defined set of general test cases is distributed to each of the independent runtime environments for execution testing, and raw test data generated by each of the target programming languages ​​during the execution of the set of general test cases is collected. The original test data are normalized according to the preset standardized mapping rules to construct a standard feature vector corresponding to each target programming language. Based on a preset feature space difference measurement model, spatial distance is calculated for each of the standard feature vectors to obtain evaluation results for multiple target programming languages.

2. The method of claim 1, wherein, The method further includes: Extract common low-level instruction abstraction structures from the first type of programming languages ​​and the second type of programming languages; Based on the underlying instruction abstraction structure, and combined with preset abnormal attack injection test vectors, a cross-language and compatible general test case set is constructed.

3. The method according to claim 1, characterized in that, The step of normalizing each of the original test data according to a preset standardized mapping rule to construct a standard feature vector corresponding to each of the target programming languages ​​includes: Extract resource consumption indicators, function execution status, and security defense response characteristics from each of the original test data; Using a preset dimension conversion mapping table, the resource consumption index, function execution status, and security defense response characteristics are converted into corresponding numerical sequences with unified dimensions. Multiple numerical sequences are combined and concatenated to construct a standard feature vector corresponding to each target programming language.

4. The method according to claim 3, characterized in that, The step of using a preset dimension conversion mapping table to convert the resource consumption indicators, function execution status, and security defense response characteristics into corresponding numerical sequences with unified dimensions includes: Obtain the execution code logic complexity parameters corresponding to each of the original test data; Based on the execution code logic complexity parameter, determine the normalized penalty weight of the resource consumption indicator; Based on the dimension conversion mapping table carrying the normalized penalty weight, the resource consumption index is converted to obtain the first target dimension value. The functional execution status and security defense response characteristics are converted into dimensions using the aforementioned dimension conversion mapping table to obtain the second target dimension value. The first target dimension value and the second target dimension value are fused to obtain a corresponding numerical sequence with unified dimension.

5. The method according to claim 1, characterized in that, The preset feature space difference measurement model calculates the spatial distance for each standard feature vector to obtain evaluation results for various target programming languages, including: In the multidimensional feature space, determine the first standard feature vector corresponding to the first type of programming language and the second standard feature vector corresponding to the second type of programming language; Calculate the feature aggregation distance between the first standard feature vector and the second standard feature vector; Based on the feature aggregation distance, the performance difference between the first type of programming language and the second type of programming language is quantitatively evaluated, and an evaluation result for the target programming language is generated based on the performance difference.

6. The method according to claim 5, characterized in that, The step of quantifying and evaluating the performance difference between the first type of programming language and the second type of programming language based on the feature aggregation distance, and generating an evaluation result for the target programming language based on the performance difference, includes: Obtain the total number of test tasks contained in the general test case set; The baseline distance threshold is determined based on the total number of test tasks. The feature aggregation distance is compared with the baseline distance threshold, and the quantization score between the first type of programming language and the second type of programming language is determined based on the comparison result. The quantization score is then added as a performance difference to the evaluation result of the target programming language.

7. The method according to claim 1, characterized in that, The method further includes: Extract target feature dimensions that indicate performance weaknesses from the evaluation results; Match language-level optimization suggestions based on the target feature dimension from a preset historical optimization rule base; The underlying optimization suggestions for the language are added to the evaluation results of the target programming language.

8. A comparative evaluation device for programming languages, characterized in that, The device includes: An acquisition module is used to acquire multiple target programming languages ​​to be evaluated, wherein the target programming languages ​​include at least a first type of programming language and a second type of programming language; A module is established to create a corresponding independent runtime environment for each of the target programming languages. The testing module is used to distribute a preset set of general test cases to each of the independent runtime environments for execution testing, and to collect raw test data generated by each of the target programming languages ​​during the execution of the set of general test cases. The construction module is used to normalize each of the original test data according to a preset standardized mapping rule, and construct a standard feature vector corresponding to each of the target programming languages. The processing module is used to calculate the spatial distance of each of the standard feature vectors based on a preset feature space difference measurement model, so as to obtain the evaluation results of multiple target programming languages.

9. A computer device, characterized in that, include: A processor and a memory, the processor being configured to execute a comparison and evaluation program of a programming language stored in the memory, to implement the comparison and evaluation method of the programming language according to any one of claims 1-7.

10. A storage medium, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the comparative evaluation method of the programming languages ​​according to any one of claims 1-7.

Images