KNN-based software test scope assessment method and apparatus

Abstract

This application provides a KNN-based method and apparatus for software test scope assessment. The method includes: acquiring information on each first call chain affected by a code modification of a target software platform; determining information on the second call chains for each automated test scenario corresponding to each interface test case of the target software platform; and using the KNN model to determine all automated test scenarios corresponding to the code modification to generate a test scope assessment report for the code modification of the target software platform. This application can effectively improve the automation and intelligence of software test scope assessment, and can effectively improve the accuracy and validity of the software test scope assessment results, thereby improving the reliability and efficiency of testing based on the software test scope assessment results.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method and apparatus for evaluating the scope of software testing based on KNN. Background Technology

[0002] For microservice architecture software platforms such as cloud blockchain platforms, business architects design software platform products primarily from the perspective of product and business requirements. The scope of evaluation is also based on business dependencies, especially for the design of basic functions, where the impact is much greater. Even an adjustment to a single interface can affect other platforms, so a holistic evaluation and testing scope is necessary. Developers typically use development tools (Eclipse or an IDE) to trace the interfaces affected by method modifications. While this method is accurate for evaluating the modifications made to the service itself, for microservice architectures, where services communicate via Feign and each microservice is managed by a separate team, there are issues with low communication efficiency and inaccurate evaluations.

[0003] Currently, functional testing primarily assesses the test scope of a software platform based on requirements design documents, developers' and testers' familiarity with the cloud platform, and the resulting test case scope. Test cases are then supplemented during the testing process. However, operational issues frequently arise in production, and analysis of these issues reveals that inaccurate test case or test scope assessments are a major contributing factor. Therefore, there is an urgent need to design a method that can improve the automation of the software test scope assessment process and enhance the accuracy of the assessment results. Summary of the Invention

[0004] In view of this, embodiments of this application provide a method and apparatus for evaluating the scope of software testing based on KNN, in order to eliminate or improve one or more defects existing in the prior art.

[0005] One aspect of this application provides a KNN-based method for evaluating software test scope, including:

[0006] Obtain information on each first call chain affected by this code modification on the target software platform;

[0007] Determine the second call chain information for each automated test scenario corresponding to each interface test case of the target software platform, wherein both the first call chain information and the second call chain information contain the identifier of each interface corresponding to the call chain;

[0008] Based on the information of each of the first call chains and each of the second call chains, the KNN model is used to determine all automated test scenarios corresponding to this code modification in order to generate a test scope assessment report for this code modification of the target software platform.

[0009] In some embodiments of this application, obtaining the information of each first call chain affected by the current code modification of the target software platform includes:

[0010] Obtain the call chain data of the target software platform to generate the corresponding call chain model, wherein each of the call chain data contains the identifier of each interface corresponding to the call chain;

[0011] Determine the correspondence between each method and each interface in the modified code of the target software platform to generate the corresponding code tree structure model;

[0012] Based on the interfaces in the call chain model and the code tree structure model, a corresponding call chain and code tree structure model are generated. The call chain and corresponding interface identifiers affected by the current code modification of the target software platform are searched from the call chain and code tree structure model to obtain the corresponding information of each first call chain.

[0013] In some embodiments of this application, obtaining the call chain data of the target software platform to generate the corresponding call chain model includes:

[0014] Retrieves all current call chain data of the target software platform from a preset database, wherein the call chain data in the database is collected in advance by an open-source call chain client;

[0015] The data from each of the retrieved call chains is deduplicated.

[0016] A call chain model is generated using the deduplicated call chain data, which is then used to store the correspondence between each call chain and multiple interfaces.

[0017] In some embodiments of this application, determining the correspondence between each method and each interface corresponding to the modified code of the target software platform to generate a corresponding code tree structure model includes:

[0018] The code comparison interface of the code repository is used to determine each line of code that has been modified in this instance for the target software platform;

[0019] The open-source technology corresponding to the code repository is used to determine the correspondence between the methods and interfaces corresponding to each code, so as to generate a code tree structure model for storing the correspondence between each method of the code and multiple interfaces.

[0020] In some embodiments of this application, determining the second call chain information for each automated test scenario corresponding to each interface test case of the target software platform includes:

[0021] Receive automated test scenario data representing different business scenarios, wherein the automated test scenario data is obtained in advance based on various interface test cases of the target software platform, and the interface test cases include: interface parameters and return values;

[0022] Based on the correspondence between each of the aforementioned business scenarios and multiple interfaces, corresponding test cases and interface test case models are generated.

[0023] Based on the call chain model and the interfaces in the test cases and interface use case models, the correspondence between the business scenarios and the call chain data is found to obtain the second call chain information corresponding to each business scenario. The second call chain information is used to store the correspondence between the business scenarios, call chains and the identifiers of the corresponding interfaces.

[0024] In some embodiments of this application, the step of using a KNN model to determine all automated test scenarios corresponding to the current code modification based on each of the first call chain information and each of the second call chain information to generate a test scope assessment report for the current code modification of the target software platform includes:

[0025] The first call chain information and the second call chain information are input into a preset KNN model, so that the KNN model calculates the distance values ​​between the interfaces and business scenarios in the first call chain information and the second call chain information respectively based on the preset Euclidean algorithm, so as to obtain a distance set;

[0026] Arrange the distance values ​​in the distance set in ascending order, and select the first K distance values ​​from the arranged distance set, where K is a positive integer equal to or greater than 1;

[0027] The second call chain information corresponding to the first K distance values ​​is determined as the target second call chain information, and the automated test scenario corresponding to each of the target second call chain information is used as the test scope evaluation data for the current code modification of the target software platform.

[0028] Based on the test range assessment data, a corresponding test range assessment report is generated and output.

[0029] In some embodiments of this application, before determining all automated test scenarios corresponding to the current code modification using a KNN model based on each of the first call chain information and each of the second call chain information to generate a test scope assessment report for the current code modification of the target software platform, the method further includes:

[0030] Based on the historical call chain data of the target software platform and the correspondence between each method and each interface corresponding to the historical modified code, the information of each first historical call chain affected by the historical modified code of the target software platform is obtained.

[0031] Determine the second historical call chain information involved in the automated test scenarios corresponding to each interface test case for the target software platform;

[0032] The KNN model is trained using the first historical call chain information and the second historical call chain information.

[0033] Another aspect of this application provides a KNN-based software test range evaluation apparatus, comprising:

[0034] The code impact data acquisition module is used to acquire information about the first call chains affected by this code modification on the target software platform.

[0035] The test scenario data acquisition module is used to determine the second call chain information of each automated test scenario corresponding to each interface test case of the target software platform. The first call chain information and the second call chain information both contain the identifier of each interface corresponding to the call chain.

[0036] The test scope automatic assessment module is used to determine all automated test scenarios corresponding to this code modification based on each of the first call chain information and each of the second call chain information, using the KNN model, so as to generate a test scope assessment report for this code modification for the target software platform.

[0037] A third aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the KNN-based software test range evaluation method.

[0038] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the described KNN-based software test range evaluation method.

[0039] The KNN-based software test scope assessment method provided in this application obtains information on each first call chain affected by the current code modification of the target software platform; determines information on the second call chain for each automated test scenario corresponding to each interface test case of the target software platform, wherein both the first and second call chain information contain the identifiers of each interface corresponding to the call chain; based on each of the first and second call chain information, a KNN model is used to determine all automated test scenarios corresponding to the current code modification to generate a test scope assessment report for the current code modification of the target software platform. This method can effectively improve the automation and intelligence of software test scope assessment, and can effectively improve the accuracy and validity of the software test scope assessment results, thereby improving the reliability and efficiency of testing based on the software test scope assessment results. Furthermore, accurate assessment can be performed in the early stages of testing, allowing sufficient time for business architecture, development, and testing for discussion, analysis, development, and testing, ensuring a good user experience for the platform. When submitting development for testing, assessing the risks within the test scope allows for a more accurate determination of the development plan, especially for underlying foundational services. Adding or removing interface fields becomes much more cautious; in previous development designs, there was a reluctance to reduce interface fields, leading to increasingly bloated interface designs that eventually became unmaintainable. Such technical debt could only be addressed through platform refactoring, resulting in platform instability and a waste of company development resources. In the early stages of testing, assessing the risks within the test scope allows for the early revision of test cases. Targeted and detailed design and testing of relevant test cases can avoid relying on experience or familiarity with the system for test case scope assessment. By making the assessment tool-based and data-driven, the quality of project deployment is improved, and production environment maintenance issues are reduced.

[0040] Additional advantages, objectives, and features of this application will be set forth in part in the description which follows, and will in part become apparent to those skilled in the art upon review of the following description, or may be learned by practice of the application. The objectives and other advantages of this application can be realized and obtained by means of the structures specifically pointed out in the specification and drawings.

[0041] Those skilled in the art will understand that the purposes and advantages that can be achieved with this application are not limited to those specifically described above, and that the above and other purposes that this application can achieve will be more clearly understood from the following detailed description. Attached Figure Description

[0042] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, do not constitute a limitation thereof. The components in the drawings are not drawn to scale but are merely for illustrating the principles of this application. For ease of illustration and description of certain parts of this application, corresponding portions in the drawings may be enlarged, i.e., may appear larger relative to other components in an exemplary device actually manufactured according to this application. In the drawings:

[0043] Figure 1 This is a schematic diagram of the first process of a KNN-based software test range evaluation method in one embodiment of this application.

[0044] Figure 2 This is a schematic diagram of the second process of the KNN-based software test range evaluation method in one embodiment of this application.

[0045] Figure 3 This is a schematic diagram of the structure of a KNN-based software testing range evaluation device in another embodiment of this application.

[0046] Figure 4 This is a schematic diagram illustrating the overall project development process at the current R&D center.

[0047] Figure 5 This is a detailed flowchart of the KNN-based software test scope assessment method used in an application example of this application. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and their descriptions are used to explain this application, but are not intended to limit it.

[0049] It should also be noted that, in order to avoid obscuring this application with unnecessary details, only the structures and / or processing steps closely related to the solution according to this application are shown in the accompanying drawings, while other details that are not closely related to this application are omitted.

[0050] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0051] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0052] In the following description, embodiments of the present application will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0053] In one or more embodiments of this application, the target software platform refers to the software platform for which the software testing scope assessment is to be performed, and the software platform can be a financial service platform adopting a microservice architecture, such as a cloud blockchain platform.

[0054] Specifically, (1) all services on the CloudChain platform are currently developed based on the Spring Boot microservice approach. (2) all microservices have been integrated with the Cloud Intelligence platform, and all call chain information is basically complete. (3) The CloudChain continuous delivery platform has implemented functions such as code branching, code scanning, and unit testing for each project. (4) CloudChain already has the CloudJie platform, and has begun to develop test case, interface test case and interface relationship management platform, automated testing platform, and stress testing platform. In a specific example, the software test scope assessment method based on KNN provided in this application embodiment mainly functions to construct a KNN model (also known as a test scope risk assessment model) for test scope assessment based on artificial intelligence using the system data generated in (1) to (4) above, and to conduct risk assessment of the test scope of each project on the CloudChain platform.

[0055] In one or more embodiments of this application, continuous delivery (CD) is a software engineering approach that ensures the software product is produced within a short cycle, guaranteeing that the software remains stable and continuously available for release. Its goal is to make software building, testing, and release faster and more frequent. This approach can reduce software development costs and time, and mitigate risks.

[0056] In one or more embodiments of this application, the test scope refers to the range of business function boundaries affected by this project modification. Testers need to write test cases and evaluate whether to conduct performance testing and security testing based on this scope.

[0057] In one or more embodiments of this application, test scope assessment can also be called test scope risk assessment: it refers to a method of assessing the test scope and test cases, mainly to assess whether the scope is fully covered, whether the test cases are detailed, and whether the security tests are complete. Through risk assessment, the quality of this project is ensured.

[0058] In one or more embodiments of this application, a call chain is generated as follows: During the process of a system completing a business call, the call information (time, interface, layer, result) between services is logged. Then, all the logged data are connected into a tree-like chain to create a call chain. The tracking system analyzes and processes the log information generated during the process, reconstructs the complete end-to-end execution of the business call process, and performs statistical analysis according to different dimensions. This identifies abnormal service calls, enabling rapid analysis and delineation of the abnormal service. Simultaneously, it allows for statistical analysis of system performance bottlenecks.

[0059] In one or more embodiments of this application, the KNN model refers to the K-Nearest Neighbors algorithm: KNN classifies objects by searching for the K most similar instances, i.e., the K neighbors, throughout the training set and assigning a common output variable to all of these K instances.

[0060] In one or more embodiments of this application, the distance in the Euclidean algorithm, also known as Euclidean distance, is a commonly used definition of distance, referring to the true distance between two points in m-dimensional space, or the natural length of a vector, i.e., the distance from that point to the origin. In two-dimensional and three-dimensional space, the Euclidean distance is the actual distance between two points, and the mathematical formula is: sqrt((x1-x2)^+(y1-y2)^).

[0061] For service modifications made during development, the affected interfaces are typically traced using development tools (Eclipse or an IDE). While this method is accurate for evaluating the modifications themselves, it suffers from low communication efficiency and inaccurate assessments in microservice architectures where services communicate via Feign and each microservice is managed by a separate team. This application first considers analyzing the code and call chains of all platform services to construct a relationship model between code and call chains. Analysis of this model significantly improves both speed and accuracy.

[0062] Furthermore, current functional testing primarily assesses the test scope based on requirements design documents, developers' and testers' familiarity with the cloud platform, and other factors. Test cases are then added during the testing process. However, operational issues frequently arise in production, and analysis of these issues reveals that inaccurate test case or test scope assessments are a major contributing factor. This application addresses these issues by using artificial intelligence to train models of code modification points, call chains, and test cases to improve the accuracy of test scope assessment for the platform project.

[0063] The following examples will provide a detailed description.

[0064] This application provides a KNN-based software test scope evaluation method, see [link to relevant documentation]. Figure 1 The KNN-based software test range assessment method, which can be executed by the KNN-based software test range assessment device, specifically includes the following:

[0065] Step 100: Obtain information on each first call chain affected by this code modification on the target software platform.

[0066] It is understood that each of the first call chain information is used to store the identifier of a call chain affected by this code modification of the target software platform and the identifier of the interface corresponding to the call chain.

[0067] In other words, step 100 requires finding the call chain data affected by this code modification in all the current call chain data of the target software platform, and forming each first call chain information according to the correspondence between the call chain identifier and the interface identifier in these call chain data affected by this code modification.

[0068] Step 200: Determine the second call chain information for each automated test scenario corresponding to each interface test case of the target software platform, wherein both the first call chain information and the second call chain information contain the identifiers of each interface corresponding to the call chain.

[0069] It is understood that each of the first call chain information is used to store the correspondence between the identifier of an automated test scenario (i.e., a business scenario) of the target software platform, the identifier of the call chain involved in the automated test scenario, and the identifier of the interface corresponding to the call chain identifier.

[0070] In other words, step 200 requires searching through all the current call chain data of the target software platform to find the call chain data that involves each of the current automated test scenarios of the target software platform, and forming each of the second call chain information based on the correspondence between the call chain identifier and the interface identifier in these call chain data that involve each of the current automated test scenarios of the target software platform, as well as the identifier of the corresponding automated test scenario.

[0071] Step 300: Based on the information of each of the first call chains and each of the second call chains, use the KNN model to determine all automated test scenarios corresponding to this code modification in order to generate a test scope evaluation report for this code modification of the target software platform.

[0072] In step 300, the KNN model is used to output the distance values ​​between the interfaces and business scenarios in each of the first call chain information and each of the second call chain information according to the input first call chain information and each of the second call chain information, so as to obtain a distance set.

[0073] As can be seen from the above description, the KNN-based software test scope assessment method provided in this application can effectively improve the automation and intelligence of software test scope assessment, and can effectively improve the accuracy and validity of software test scope assessment results, thereby improving the reliability and efficiency of testing based on the software test scope assessment results.

[0074] To further improve the effectiveness and reliability of obtaining the first call chain information, a KNN-based software test range evaluation method is provided in this application embodiment, see [link to relevant documentation]. Figure 2 Step 100 in the KNN-based software test scope evaluation method specifically includes the following:

[0075] Step 110: Obtain the call chain data of the target software platform to generate the corresponding call chain model, wherein each call chain data contains the identifier of each interface corresponding to the call chain.

[0076] Step 120: Determine the correspondence between each method and each interface in the modified code of the target software platform to generate the corresponding code tree structure model.

[0077] Step 130: Based on the interfaces in the call chain model and the code tree structure model, generate corresponding call chains and code tree structure models, and search for the call chains and corresponding interface identifiers affected by the current code modification of the target software platform in the call chain and code tree structure models to obtain the corresponding information of each first call chain.

[0078] It is understood that the call chain model is a data model used to store the correspondence between the call chain identifier and the identifiers of multiple interfaces contained in each of the call chain data, such as a key-value distributed database.

[0079] The code tree structure model is a data model used to store the correspondence between each method and each interface corresponding to each line of code modified in this target software platform. Since there is a dependency relationship between the codes, the relationship between each code can be represented by a tree structure. In each code node of the code tree structure, the correspondence between the method corresponding to that code and the identifiers of multiple interfaces is recorded.

[0080] The step of generating corresponding call chains and code tree structures based on the interfaces in the call chain model and the code tree structure model specifically means that, since both the call chain model and the code tree structure model contain interface identifiers, they can be associated with the same interface identifiers to form call chain and code tree structure models. These call chain and code tree structure models are used in some code nodes of their code tree structure to store, in addition to storing the identifiers of the methods corresponding to the code and the correspondence between the identifiers of the multiple interfaces corresponding to those methods, the correspondence between the identifiers of the call chains corresponding to these interfaces.

[0081] Then, by using the call chain and code tree structure model, we can find the code nodes that, in addition to storing the identifier of the method corresponding to the code and the correspondence between the identifiers of the multiple interfaces corresponding to the method, also store the correspondence between the identifiers of the call chains corresponding to these interfaces. The correspondence between the call chain identifiers and interface identifiers corresponding to these nodes is then determined as different first call chain information.

[0082] Specifically, in step 130, the data R obtained in step 120 is first looped, and Yn is looped again in the first loop. If Rxn = Yxn, the result is recorded as Zn = {(Xn,Yn)}, and the loop is exited to continue the next loop. When R has completely looped, the call chain and code tree structure model are obtained. Based on this model, it is possible to calculate which call chains are affected by the current code modification, and this call chain information is stored in the MySQL database.

[0083] To further improve the efficiency and reliability of acquiring various call chain data of the target software platform, in the software test scope evaluation method based on KNN provided in this application embodiment, step 110 of the software test scope evaluation method based on KNN specifically includes the following:

[0084] Step 111: Retrieve all current call chain data of the target software platform from the preset database, wherein the call chain data in the database is collected in advance by an open-source call chain client;

[0085] Step 112: Perform deduplication processing on each of the retrieved call chain data;

[0086] Step 113: Generate a call chain model using the deduplicated call chain data, so that the call chain model can be used to store the correspondence between each call chain and multiple interfaces.

[0087] Specifically, all Spring Boot microservices on the platform are now integrated with the Cloud Intelligence Platform. The Cloud Intelligence Platform collects call chain information using the Skywalking agent (an open-source call chain client). All call chain information is stored in an Elasticsearch (ES) database. By deduplicating the call chain information, all business call chains on the platform are identified, denoted as Yn = {X1, X2, X3, ..., Xn}, where Yn represents a unique call chain on the platform, and Xn represents the information of each interface in the call chain. This prepares the platform for a call chain and code tree structure model.

[0088] To further improve the efficiency and reliability of obtaining target software platform data to determine the modified code, a KNN-based software test scope evaluation method provided in this application embodiment includes the following specific steps in step 120:

[0089] Step 121: Determine each line of code modified in this instance for the target software platform based on the comparison interface of the code repository.

[0090] Step 122: Using the open-source technology corresponding to the code repository, determine the correspondence between the methods and interfaces corresponding to each piece of code, so as to generate a code tree structure model for storing the correspondence between each method of the code and multiple interfaces.

[0091] Specifically, when this project is submitted for testing, based on the code branch information of the continuous delivery platform, the code of all services of the entire platform is pulled from GitLab (code repository). The comparison interface of GitLab (code repository) is called (using the open-source gitlab4j-api JAR package) to compare this branch with the master branch, obtaining every line of code for all modifications. Using Java-all-call-graph (an open-source technology on GitLab), the relationships between methods and interfaces are obtained, resulting in a collection of methods and interfaces as objects. This is denoted as R = {X1, X2, X3, ..., Xn}, where R represents a result set and Xn represents each interface information, preparing for the call chain and code tree structure model.

[0092] To further improve the efficiency and reliability of obtaining second call chain information, in a KNN-based software test scope evaluation method provided in this application embodiment, step 200 of the KNN-based software test scope evaluation method specifically includes the following:

[0093] Step 210: Receive automated test scenario data representing different business scenarios. The automated test scenario data is obtained in advance based on various interface test cases of the target software platform, and the interface test cases include: interface parameters and return values.

[0094] Step 220: Generate corresponding test cases and interface test case models based on the correspondence between each of the aforementioned business scenarios and multiple interfaces.

[0095] It is understood that the test cases and interface test case models are a data model used to store the correspondence between each of the business scenarios and multiple interfaces, such as a key-value distributed database.

[0096] Step 230: Based on the call chain model and the interfaces in the test case and interface use case model, find the correspondence between the business scenario and the call chain data to obtain the second call chain information corresponding to each business scenario. The second call chain information is used to store the correspondence between the business scenario, the call chain and the identifier of the corresponding interface.

[0097] Specifically, the continuous delivery platform collects all interface information (including parameters, return values, and platform-generated unique codes) and stores it on the Yunjie platform. Testers first write interface test cases, which include detailed parameters and return values ​​of the interface. Then, based on these interface test cases, they write automated test scenarios. Each automated test scenario represents a business scenario, denoted as Dn = {X1, X2, X3, ..., Xn}. Here, Xn represents each piece of interface information, and Dn represents a specific business scenario. Based on the information from step 1, Dn is looped through first, then Yn is looped through to obtain the relationship (Xn, Yn, Dn), which is denoted as Fn.

[0098] To further improve the accuracy and reliability of generating a test scope assessment report for the current code modification of the target software platform, a KNN-based software test scope assessment method provided in this application embodiment includes the following specific steps in step 300:

[0099] Step 310: Input the first call chain information and the second call chain information into a preset KNN model, so that the KNN model calculates the distance values ​​between the interface and the business scenario in the first call chain information and the second call chain information respectively based on the preset Euclidean algorithm, so as to obtain a distance set.

[0100] Step 320: Sort the distance values ​​in the distance set from smallest to largest, and select the first K distance values ​​from the sorted distance set, where K is a positive integer equal to or greater than 1.

[0101] Step 330: Determine the second call chain information corresponding to the first K distance values ​​as the target second call chain information, and use the automated test scenario corresponding to each of the target second call chain information as the test scope evaluation data for this code modification of the target software platform.

[0102] Step 340: Generate a corresponding test range assessment report based on the test range assessment data, and output the test range assessment report.

[0103] Specifically, the test report is generated based on the K-nearest neighbor algorithm. First, based on the data Fn and Zn from steps 3 and 4, the distance r is calculated by substituting Xn and Yn of Fn and Zn into the Euclidean formula sqrt((x1-x2)^+(y1-y2)^). This results in a set of r, denoted as L=[r1,r2,r3,...,rn]. The set L is then sorted in reverse order, and K values ​​are selected. The value of K is determined based on multiple evaluation reports. For the cloud chain platform, when K=10, it can basically cover all test scenarios. After continuous optimization, the value of K tends to decrease. At this point, a test scope risk assessment report is automatically generated based on the content of Fn.

[0104] To further improve the effectiveness and reliability of the KNN model, in a software test range evaluation method based on KNN provided in this application embodiment, the method further includes the following content before step 300 or step 100:

[0105] Step 010: Based on the historical call chain data of the target software platform and the correspondence between each method and each interface corresponding to the historical modified code, obtain the information of each first historical call chain affected by the historical modified code of the target software platform.

[0106] For example, prepare 10 sets of call chain data, denoted as Y1, Y2, ..., Y10. Each set of call chain data contains the interface X4, but the order of the X4 interface in each set of call chain data is different. Prepare one set of code modification data, denoted as R = {X2, X4, X8}, indicating that the modified code only affects interfaces X2, X4, and X8. Combine the R data and the data from step 1 in a loop to obtain Zn = {(Pxn, Pyn)}, where Pxn and Pyn represent the node position data of the current interface in the current call chain, and Pyn represents the sequence number of the current call chain in the global call chain.

[0107] Step 020: Determine the second historical call chain information involved in the automated test scenarios corresponding to each interface test case of the target software platform.

[0108] For example, obtain all business scenario data from the Yunjie platform, denoted as Dn = {X1, X2, X3, ..., Xn}. Based on the call chain data from step 1, first loop through Dn, then loop through Yn to obtain Fn = {Pxn, Pyn, Dn}, where Pxn represents the node position of Xn in business scenario Dn, and Pyn represents the associated call chain sequence number.

[0109] Step 030: Train the KNN model using the first historical call chain information and the second historical call chain information.

[0110] Calculate the distance r using Euclidean formula sqrt((x1-x2)^+(y1-y2)^), which will result in a set R, denoted as L=[r1,r2,r3,...,rn]. Then, sort the set L in reverse order.

[0111] From a software perspective, based on the aforementioned embodiments of the KNN-based software test scope evaluation method, this application also provides an embodiment of a KNN-based software test scope evaluation apparatus for implementing the aforementioned KNN-based software test scope evaluation method. See [link to relevant documentation]. Figure 3 The KNN-based software testing scope evaluation device specifically includes the following components:

[0112] The code impact data acquisition module 10 is used to acquire information about each first call chain affected by this code modification on the target software platform;

[0113] The test scenario data acquisition module 20 is used to determine the second call chain information of each automated test scenario corresponding to each interface test case of the target software platform, wherein the first call chain information and the second call chain information both contain the identifier of each interface corresponding to the call chain.

[0114] The test scope automatic evaluation module 30 is used to determine all automated test scenarios corresponding to this code modification based on each of the first call chain information and each of the second call chain information, using the KNN model, so as to generate a test scope evaluation report for this code modification for the target software platform.

[0115] The KNN-based software test range assessment device provided in this application can be used to execute the processing flow of the embodiment of the KNN-based software test range assessment method in the above embodiments. Its functions will not be repeated here, but can be referred to the detailed description of the above embodiment of the KNN-based software test range assessment method.

[0116] The KNN-based software test scope assessment device can perform the KNN-based software test scope assessment on a server, or in another practical application scenario, all operations can be completed on the client device. The choice can be made based on the processing power of the client device and the limitations of the user's usage scenario. This application does not impose any limitations on this. If all operations are completed on the client device, the client device may further include a processor for the specific processing of the KNN-based software test scope assessment.

[0117] The aforementioned client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission. The server may include a server on the task scheduling center side; in other implementation scenarios, it may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, a server cluster consisting of multiple servers, or a distributed server structure.

[0118] The server and the client device can communicate using any suitable network protocol, including those not yet developed as of the date of this application. Such network protocols may include, for example, TCP / IP, UDP / IP, HTTP, HTTPS, etc. Furthermore, such network protocols may also include RPC (Remote Procedure Call Protocol) and REST (Representational State Transfer Protocol) protocols used on top of the aforementioned protocols.

[0119] As can be seen from the above description, the KNN-based software test scope assessment device provided in this application can effectively improve the automation and intelligence of software test scope assessment, and can effectively improve the accuracy and validity of software test scope assessment results, thereby improving the reliability and efficiency of testing based on the software test scope assessment results.

[0120] To further illustrate this solution, this application also provides a specific application example of a KNN-based software test scope evaluation method, which includes the following:

[0121] (I) Overall Project Development Process of the R&D Center

[0122] See Figure 4 This represents the current overall project development process at the R&D center, showing the stage of intelligent testing and risk assessment within the overall development process. It illustrates the project development workflow.

[0123] The current development process involves business architecture design, detailed design and test scope assessment, code writing and test case writing, development completion and submission for testing, and then deployment.

[0124] After the development and testing phases are completed, a risk assessment is conducted using the continuous delivery platform's capabilities. The assessment results are then fed back to the business architect, development engineers, and test engineers. Following this continuous, iterative process, once the overall project is deemed risk-free, it will be deployed.

[0125] (II) KNN-based software test scope assessment method

[0126] See Figure 5 It details all the process nodes of the intelligent testing scope risk assessment, including the corresponding technical architecture.

[0127] The storage of call chain information in the cloud intelligence platform in an Elasticsearch (ES) database and the use of a GitLab (code repository) comparison interface (using the gitlab4j-api open-source JAR package) are common practices in existing technologies, and this application also draws on these methods in its application examples. One improvement in this application example is the use of Java-all-call-graph technology to obtain the relationships between methods and interfaces, resulting in a collection of methods and interfaces as objects. This is part of the call chain and code tree structure model, as well as the use of the K-Nearest Neighbor (KNN) algorithm for the test case and interface relationship model.

[0128] Step 1: Data Source for the Call Chain Model: Currently, all Spring Boot (microservice development technology) microservices on the platform have been integrated with the Cloud Intelligence Platform. The Cloud Intelligence Platform collects call chain information through the Skywalking Agent (an open-source call chain client), storing all call chain information in an Elasticsearch (ES) database. By deduplicating the call chain information, all business call chains on the platform are identified, denoted as Yn = {X1, X2, X3, ..., Xn}, where Yn represents a unique call chain on the platform, and Xn represents the information of each interface on the call chain, preparing for the call chain and code tree structure model.

[0129] Step 2: Code Tree Structure Model: When this project is submitted for testing, based on the code branch information of the continuous delivery platform, the code of all services of the entire platform is pulled from GitLab (code repository). The comparison interface of GitLab (code repository) is called (using the gitlab4j-api open-source JAR package) to compare this branch with the master branch, obtaining every line of code of all modifications. Using Java-all-call-graph (an open-source technology on GitLab), the relationship between methods and interfaces is obtained, resulting in a collection of methods and interfaces as objects. This is denoted as R = {X1, X2, X3, ..., Xn}, where R represents a result set and Xn represents each interface information, preparing for the call chain and code tree structure model.

[0130] Step 3: Generate the call chain and code tree structure model: First, loop through the data R obtained in Step 2. Continue looping through Yn in the first loop. If Rxn = Yxn, record the result Zn = {(Xn,Yn)}, then exit this loop and continue to the next loop. When R has completely looped, the call chain and code tree structure model is obtained. Based on this model, it is possible to calculate which call chains are affected by the current code modification. Store this call chain information in the MySQL database as input data for Step 5.

[0131] Step 4: Test Case and Interface Test Case Model: The continuous delivery platform collects all interface information (including parameters, return values, and platform-generated unique codes) and stores it on the YunJie platform. Testers first write interface test cases, which include detailed parameters and return values ​​of the interface. Then, based on these interface test cases, automated test scenarios are written. Each automated test scenario represents a business scenario, denoted as Dn = {X1, X2, X3, ..., Xn}. Here, Xn represents each piece of interface information, and Dn represents a specific business scenario. Based on the information from Step 1, Dn is looped through first, then Yn is looped through to obtain the relationship (Xn, Yn, Dn), denoted as Fn, which serves as the input data for Step 5.

[0132] Step 5: Generate a test scope risk assessment report: A test report is generated based on the K-nearest neighbor algorithm. First, based on the data Fn and Zn from steps 3 and 4, the distance r is calculated by substituting Xn and Yn of Fn and Zn into the Euclidean formula sqrt((x1-x2)^+(y1-y2)^). This results in a set R, denoted as L = [r1, r2, r3, ..., rn]. The set L is then sorted in reverse order, and K values ​​are selected. The value of K is determined based on multiple assessment reports. For the cloud chain platform, setting K = 10 generally covers all test scenarios. Through continuous optimization, the value of K tends to decrease. At this point, a test scope risk assessment report is automatically generated based on the content of Fn.

[0133] (III) KNN Model Training

[0134] Step 1: Prepare 10 sets of call chain data, denoted as Y1, Y2, ..., Y10. Each set of call chain data contains the interface X4, but the order of the X4 interface in each set of call chain data is different.

[0135] Step 2: Prepare a set of code modification data, denoted as R = {X2, X4, X8}, indicating that the modified code only affects the interfaces X2, X4, and X8.

[0136] Step 3: Combine the R data and the data from Step 1 in a loop to obtain Zn = {(Pxn, Pyn)}, where Pxn and Pyn represent the node position data of the current call chain of the current interface, and Pyn represents the sequence number of the current call chain in all global call chains.

[0137] Step 4: Obtain all business scenario data from the Yunjie platform, denoted as Dn = {X1, X2, X3, ..., Xn}. Simultaneously, based on the call chain data from Step 1, loop through Dn first, then loop through Yn to obtain Fn = {Pxn, Pyn, Dn}, where Pxn represents the node position of Xn in business scenario Dn, and Pyn represents the associated call chain sequence number.

[0138] Step 5: Calculate the distance r using the Euclidean formula sqrt((x1-x2)^+(y1-y2)^), which will result in a set R, denoted as L=[r1,r2,r3,...,rn]. Reverse the order of set L.

[0139] Therefore, in the actual use of the method, for business scenarios where the deviation distance of individual interfaces is greater than 10, it will also be displayed to prompt the testers, who can then selectively conduct tests according to the situation.

[0140] This greatly improves the accuracy of test scope assessment while reducing the number of specialized personnel required for testing.

[0141] Due to the large number of interfaces on the platform, the calculation speed still needs continuous optimization to improve the efficiency of the evaluation.

[0142] In summary, this application example utilizes a code tree structure and system call chain relationship model, a test case and interface relationship model, and an interface scanning method for all services. It employs a model trained using the K-Nearest Neighbors (KNN) algorithm based on artificial intelligence. This addresses the issue of business architects being unable to accurately modify the scope during the design phase. Based on AI-based test scope risk assessment, accurate evaluation can be conducted in the early stages of testing, allowing sufficient time for business architecture, development, and testing teams to discuss, analyze, develop, and test, thus ensuring a superior user experience on the platform.

[0143] When submitting development for testing, the development plan can be more accurately determined based on the risk assessment of the test scope. In particular, the addition or reduction of interface fields in the underlying basic services is more cautious. In fact, in the previous development plan design, we dared not reduce the number of interface fields, which led to the interface design becoming more and more bloated until it became unmaintainable. Such technical debt can only be solved by platform reconstruction, which brings instability to the platform and wastes the company's development resources.

[0144] In the early stages of testing, based on the assessment of the risks to the test scope, test cases can be revised in advance. Detailed design and testing of relevant test cases can be carried out in a targeted manner, avoiding reliance on experience or familiarity with the system for test case scope assessment. The assessment basis is tool-based and data-driven, improving the quality of project launch and reducing operational and maintenance issues in the production environment.

[0145] This application also provides an electronic device (i.e., a computer device), which may include a processor, a memory, a receiver, and a transmitter. The processor is used to execute the KNN-based software test range evaluation method mentioned in the above embodiments. The processor and memory can be connected via a bus or other means, taking a bus connection as an example. The receiver can be connected to the processor and memory via wired or wireless means.

[0146] The processor can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0147] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the KNN-based software test range evaluation method in the embodiments of this application. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory, thereby implementing the KNN-based software test range evaluation method in the above method embodiments.

[0148] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0149] The one or more modules are stored in the memory, and when executed by the processor, the KNN-based software test range evaluation method in the embodiment is executed.

[0150] In some embodiments of this application, the user equipment may include a processor, a memory, and a transceiver unit. The transceiver unit may include a receiver and a transmitter. The processor, memory, receiver, and transmitter may be connected via a bus system. The memory is used to store computer instructions, and the processor is used to execute the computer instructions stored in the memory to control the transceiver unit to send and receive signals.

[0151] As one implementation method, the functions of the receiver and transmitter in this application can be implemented by transceiver circuits or dedicated transceiver chips, and the processor can be implemented by dedicated processing chips, processing circuits or general-purpose chips.

[0152] As another implementation approach, the server provided in this application embodiment can be implemented using a general-purpose computer. That is, the program code implementing the processor, receiver, and transmitter functions is stored in memory, and the general-purpose processor implements the processor, receiver, and transmitter functions by executing the code in memory.

[0153] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the aforementioned KNN-based software test scope evaluation method. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

[0154] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave.

[0155] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0156] In this application, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0157] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to the embodiments of this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A software test scope evaluation method based on KNN, characterized in that, include: Obtain information on each first call chain affected by this code modification on the target software platform; Determine the second call chain information for each automated test scenario corresponding to each interface test case of the target software platform, wherein both the first call chain information and the second call chain information contain the identifier of each interface corresponding to the call chain; Based on the information of each of the first call chains and each of the second call chains, the KNN model is used to determine all the automated test scenarios corresponding to this code modification in order to generate a test scope evaluation report for this code modification of the target software platform. Based on the information from each of the first and second call chains, the KNN model is used to determine all automated test scenarios corresponding to this code modification in order to generate a test scope assessment report for this code modification on the target software platform, including: The first call chain information and the second call chain information are input into a preset KNN model, so that the KNN model calculates the distance values ​​between the interfaces and business scenarios in the first call chain information and the second call chain information respectively based on the preset Euclidean algorithm, so as to obtain a distance set; Arrange the distance values ​​in the distance set in ascending order, and select the first K distance values ​​from the arranged distance set, where K is a positive integer equal to or greater than 1; The second call chain information corresponding to the first K distance values ​​is determined as the target second call chain information, and the automated test scenario corresponding to each of the target second call chain information is used as the test scope evaluation data for the current code modification of the target software platform. Based on the test range assessment data, a corresponding test range assessment report is generated and output.

2. The software test scope evaluation method based on KNN according to claim 1, characterized in that, The acquisition of information on each first call chain affected by this code modification of the target software platform includes: Obtain the call chain data of the target software platform to generate the corresponding call chain model, wherein each of the call chain data contains the identifier of each interface corresponding to the call chain; Determine the correspondence between each method and each interface in the modified code of the target software platform to generate the corresponding code tree structure model; Based on the interfaces in the call chain model and the code tree structure model, a corresponding call chain and code tree structure model are generated. The call chain and corresponding interface identifiers affected by the current code modification of the target software platform are searched from the call chain and code tree structure model to obtain the corresponding information of each first call chain.

3. The software test scope evaluation method based on KNN according to claim 2, characterized in that, The step of acquiring call chain data from the target software platform to generate a corresponding call chain model includes: Retrieves all current call chain data of the target software platform from a preset database, wherein the call chain data in the database is collected in advance by an open-source call chain client; The data from each of the retrieved call chains is deduplicated. A call chain model is generated using the deduplicated call chain data, which is then used to store the correspondence between each call chain and multiple interfaces.

4. The software test scope evaluation method based on KNN according to claim 2, characterized in that, The process of determining the correspondence between each method and each interface corresponding to the modified code of the target software platform, in order to generate a corresponding code tree structure model, includes: The code comparison interface of the code repository is used to determine each line of code that has been modified in this instance for the target software platform; The open-source technology corresponding to the code repository is used to determine the correspondence between the methods and interfaces corresponding to each code, so as to generate a code tree structure model for storing the correspondence between each method of the code and multiple interfaces.

5. The software test scope evaluation method based on KNN according to claim 2, characterized in that, The determination of the second call chain information for each automated test scenario corresponding to each interface test case of the target software platform includes: Receive automated test scenario data representing different business scenarios, wherein the automated test scenario data is obtained in advance based on various interface test cases of the target software platform, and the interface test cases include: interface parameters and return values; Based on the correspondence between each of the aforementioned business scenarios and multiple interfaces, corresponding test cases and interface test case models are generated. Based on the call chain model and the interfaces in the test cases and interface use case models, the correspondence between the business scenarios and the call chain data is found to obtain the second call chain information corresponding to each business scenario. The second call chain information is used to store the correspondence between the business scenarios, call chains and the identifiers of the corresponding interfaces.

6. The software test scope evaluation method based on KNN according to claim 1, characterized in that, Before generating a test scope assessment report for the code modification on the target software platform by using a KNN model to determine all automated test scenarios corresponding to this code modification based on each of the first call chain information and each of the second call chain information, the method further includes: Based on the historical call chain data of the target software platform and the correspondence between each method and each interface corresponding to the historical modified code, the information of each first historical call chain affected by the historical modified code of the target software platform is obtained. Determine the second historical call chain information involved in the automated test scenarios corresponding to each interface test case for the target software platform; The KNN model is trained using the first historical call chain information and the second historical call chain information.

7. A software test range evaluation device based on KNN, characterized in that, include: The code impact data acquisition module is used to acquire information about the first call chains affected by this code modification on the target software platform. The test scenario data acquisition module is used to determine the second call chain information of each automated test scenario corresponding to each interface test case of the target software platform. The first call chain information and the second call chain information both contain the identifier of each interface corresponding to the call chain. The test scope automatic assessment module is used to determine all automated test scenarios corresponding to this code modification based on each of the first call chain information and each of the second call chain information, using the KNN model, so as to generate a test scope assessment report for this code modification for the target software platform. Based on the information from each of the first and second call chains, the KNN model is used to determine all automated test scenarios corresponding to this code modification in order to generate a test scope assessment report for this code modification on the target software platform, including: The first call chain information and the second call chain information are input into a preset KNN model, so that the KNN model calculates the distance values ​​between the interfaces and business scenarios in the first call chain information and the second call chain information respectively based on the preset Euclidean algorithm, so as to obtain a distance set; Arrange the distance values ​​in the distance set in ascending order, and select the first K distance values ​​from the arranged distance set, where K is a positive integer equal to or greater than 1; The second call chain information corresponding to the first K distance values ​​is determined as the target second call chain information, and the automated test scenario corresponding to each of the target second call chain information is used as the test scope evaluation data for the current code modification of the target software platform. Based on the test range assessment data, a corresponding test range assessment report is generated and output.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the KNN-based software test range evaluation method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the KNN-based software test range evaluation method as described in any one of claims 1 to 6.

Images