Software test method and platform

Abstract

The invention relates to a method that enables the execution of the minimum possible number of test scenarios, selected from among test scenarios created for software developed on different platforms, in the most appropriate order, and to a remote test scenario execution platform using this method. The invention particularly relates to a software test method and platform comprising a database, an interface, a server, a virtual machine, and a virtual database and a virtual server within the virtual machine.

Description

[0001] SOFTWARE TEST METHOD AND PLATFORM

[0002] Technical field:

[0003] The invention relates to a method that enables the execution of the minimum possible number of test scenarios, selected from among test scenarios created for software developed on different platforms, in the most appropriate order, and to a remote test scenario execution platform using this method.

[0004] The invention particularly relates to a software test method and platform comprising a database, an interface, a server, a virtual machine, and a virtual database and a virtual server within the virtual machine.

[0005] State of the art:

[0006] Mobile / web application platforms that generate software tests and test scenarios and enable users to log into the platform and perform appropriate tests have become indispensable tools in the software development process. These platforms are used in order to increase software quality, improve user experience, and minimise potential errors.

[0007] One of these platforms has user-friendly interfaces that enable users to easily create and manage test scenarios. Developers and test specialists can establish a systematic test process by defining the scenarios required to test specific functions. This both saves time and ensures that tests are carried out in a more organised and effective manner.

[0008] Enabling users to log into the platform and execute test scenarios allows the software to be tested with real user experience. Such applications enable users to evaluate whether the software delivers the expected functionality by following specific scenarios. User feedback is highly valuable for the improvement of the software.

[0009] Platforms offering automation features increase efficiency by automating the execution of repetitive test scenarios. This ensures that test processes are accelerated and the likelihood of manual errors is reduced. Users can automatically execute test scenarios and carry out a rapid evaluation by reviewing the results instantly. User experience plays a critical role in the success of software. Mobile / web application platforms provide various surveys and evaluation forms to collect user feedback. This feedback is of great importance in shaping the software according to user needs and constitutes a valuable resource for development teams.

[0010] In addition, these platforms generally provide cloud-based services, enabling users to access them from anywhere. Users can log into the platform via mobile or web browsers, easily execute test scenarios, and monitor the results. This flexibility makes test processes more dynamic.

[0011] As a result, mobile / web application platforms that generate software tests and test scenarios play an important role in improving the quality of software development processes. These platforms encourage active user participation and help to understand how the software performs from the user’s perspective.

[0012] Although various suggestions and implementations have been developed in the state of the art for software test methods and platforms, these developments are insufficient. For this purpose, some applications relating to inventions developed are given below.

[0013] The patent application numbered “US10140206B2” in the state of the art provides a “pilot as a service” system designed to make software test processes more effective. This system enables the simultaneous execution of pilot tests for multiple software products via a server. In addition to a function that allows corporate and early-stage clients to register in advance, it also comprises a dashboard through which these clients can view test results. Thus, with the participation of different clients, the efficiency of proof-of-concept tests in the software development process is increased, resource utilisation is optimised, and feedback processes are accelerated.

[0014] The patent application numbered “CN113886262A” in the state of the art refers to the development of a method and a device aimed at optimising software automatic test processes. In particular, when an automatic test request is received, the test process is carried out by using a predefined RPA (Robotic Process Automation) test execution script in accordance with the software to be tested and the determined test scenario. While this method executes regular and repetitive workflows via the RPA script, artificial intelligence components also aim to increase the efficiency of automated tests by simulating manual test processes. As a result, this invention aims to improve test quality while providing time and resource savings in software test processes. The patent application numbered “CN112486812A” in the state of the art defines a cloud-based distributed framework software test method and device. Essentially, it enables the parallel execution of a large number of test cases in order to make software test processes more efficient. First, by receiving a test command, user modification code, base code, and test case information are obtained. Then, these codes are combined and compiled to form a merged code. By determining the number of test cases and the required computing resources, the test cases are executed in parallel using these resources. After the results are obtained, the software test process is completed together with the relevant system information. This method aims to increase software test efficiency by overcoming the limitations of a single computer.

[0015] The patent application numbered “EP3905051A1” in the state of the art relates to a method and a system aimed at automating software test processes. Focusing on the automatic generation of test scenarios and automation scripts, the system records application details at a micro level to create a page-navigation-based index. By using transition algorithms, it determines the required test scenarios by creating a mind map or a tree structure. In addition, it captures all basic screen features and labels and links this information to an integrated action library. As a result, it enables the automation of manual test processes in a manner that saves time and effort.

[0016] In the state of the art, there is a need for an artificial intelligence-based test method and platform that enables end users who order software to create test plans for acceptance tests performed upon delivery of developments on their products, to carry out testing, to test a certain part within the scenarios they select, and to compare with previous versions, while also providing the user with information on the minimum set of scenarios to be selected from the test scenario pool and the order in which they should be executed by using machine learning-based algorithms together with information on which software version the test will be compared against, and further enabling the user, if desired, to initiate automatic testing of these written test scenarios, to perform remote manual testing by selecting the relevant version via the platform, and to analyse and prioritise test scenarios.

[0017] As a result, due to the negative aspects described above and the inadequacy of existing solutions with respect to the subject matter, it has become necessary to carry out an improvement in the relevant technical field. The aim of the invention:

[0018] The main aim of the invention is to provide a more flexible structure and to enable the dynamic analysis of test scenarios at each software change and the recommendation of the most appropriate scenarios.

[0019] Another aim of the invention is to ensure the ordering of the selected tests by running an analysis process that determines in which order the test scenarios, which are identified as being directly and indirectly affected by code changes in the impact analysis obtained by analysing the changes between two versions, will result in the most efficient test process. Thus, the tests required for the smooth execution of test scenarios are also included in the scenario, and a test plan is delivered to the end user via the resulting final ordered list.

[0020] Another aim of the invention is to optimise test processes by analysing code changes between different versions of the software. Thus, time and cost savings are achieved by reducing the test workload in software development processes.

[0021] Another aim of the invention is to enable the analysis of how the software is perceived and evaluated by end users by providing a user-oriented test process.

[0022] Another aim of the invention is to enable the analysis and prioritisation of test scenarios by using machine learning and artificial intelligence algorithms. Thus, it demonstrates that a more specialised and focused solution is provided in the field of software test automation.

[0023] Another aim of the invention is to provide solutions that can be directly integrated into software customer acceptance processes through end-user test scenarios. Thus, with a dynamic and user-oriented test scenario creation approach, an applicable solution is provided at every stage of software development processes, and it enables the execution of remote end-to-end testing of software outputs for any profile.

[0024] Another aim of the invention is to enable the dynamic execution of test processes by analysing changes in each version of the software. Thus, software test processes are made more effective and faster.

[0025] Another aim of the invention is to increase the efficiency of the test process to be executed by means of the methods developed through machine learning and artificial intelligence studies. Thus, both the execution of only the tests that are truly required is ensured, and information on the order in which they should be executed is shared with the user, enabling potential errors to be detected in the shortest possible time.

[0026] Description of the drawings:

[0027] FIGURE-1 is the drawing of a system illustrating the subject matter of the software test method and platform of the invention.

[0028] FIGURE-2 is the drawing illustrating the diagram of the subject matter of the software test method and platform of the invention.

[0029] Reference numbers:

[0030] 1 : Database

[0031] 2: Interface

[0032] 3: Server

[0033] 4: Virtual machine

[0034] 4.1 : Virtual database

[0035] 4.2: Virtual server

[0036] 5: Git

[0037] 6: Docker unit

[0038] 100: Preparation of test scenarios by the test team

[0039] 110: Uploading of the prepared test scenarios via the interface

[0040] 120: Analysis of test scenarios by using machine learning and artificial intelligence algorithms

[0041] 121 : Scanning of all code changes between two versions marked as a version transition on the version control system

[0042] 122: Indexing of all scanned files in the database

[0043] 123: Generation of a dependency map for each file with changes by using dependencies within the file content 124: Merging the dependency maps for all files to combine all changed parts of the project on a single map

[0044] 125: Completion of impact analysis by scanning all classes on this map via a previously created source code analysis index

[0045] 126: Determination of test scenarios referenced by the analyses through the identification of analyses that may be affected

[0046] 127: Combining the determined test scenarios together with mandatory test scenarios and sending them to the test scenario prioritisation module

[0047] 130: Sending of prioritised and reduced test scenarios to the virtual machine

[0048] 131 : Prioritisation of test scenarios

[0049] 131.1 : Creation of a document topic matrix by means of a topic modelling algorithm via Docker

[0050] 131.2: Retrieval of the prepared test scenarios

[0051] 131.3: Prioritisation of test scenarios by using the document matrix via machine learning

[0052] 132: Reduction of test scenarios

[0053] 132.1 : Collection of project version information, implemented changes, and fixed defect data via Apache Lucene / Elasticsearch, source code history, and JIRA

[0054] 132.2: Reduction of test scenarios by means of machine learning algorithms using the collected data

[0055] 140: Saving of prioritised and reduced test scenarios to the virtual database

[0056] 150: Execution of testing by the virtual machine

[0057] 151 : Retrieval of the project and modules from the database by the virtual machine

[0058] 152: Retrieval of test scenarios from the virtual database

[0059] 153: Creation of automatic modules in the virtual machine

[0060] 154: Retrieval of coverage data from the virtual database

[0061] 155: Retrieval of project versions from the virtual database 156: Identification of automatic scenarios via the virtual server through the virtual database and the server

[0062] 157: Retrieval of test scenarios from the virtual database

[0063] 158: Execution of tests

[0064] 160: Generation of the test report

[0065] 170: Saving of the report to the Docker unit

[0066] 180: Identification of the requested version in Git by means of machine learning

[0067] 190: Downloading of the project from Git to the Docker unit

[0068] 200: Performance of project control in the Docker unit by means of machine learning for automatic test scenarios present in the virtual machine

[0069] 210: Display of the project downloaded to the Docker unit and the saved report on the interface via the server

[0070] Description of the invention:

[0071] The invention is, in general terms, a software test method and platform that enables the execution of the minimum possible number of test scenarios, selected from among test scenarios created for software developed on different platforms, in the most appropriate order, and that generally comprises a database (1 ), an interface (2), a server (3), a virtual machine (4) that enables the execution of tests, and a virtual database (4.1 ) and a virtual server (4.2) within the virtual machine (4), and that operates in an integrated manner with Git (5) and a docker unit (6).

[0072] The database (1 ) is an element in which the project is stored and which operates in an integrated manner with the virtual machine (4).

[0073] The interface (2) is an element in which the virtual machine (4) and the server (3) are integrated and on which the project and the report are displayed.

[0074] The server (3) is an element that serves as a connection between the docker unit (6) and the interface (2). The virtual machine (4) is an element to which reduced and prioritised test scenarios are transmitted via the interface (2), in which tests are executed, and which comprises the virtual database (4.1 ) and the virtual server (4.2).

[0075] The virtual database (4.1 ) is an element located within the virtual machine (4), in which reduced and prioritised test scenarios are stored and in which information relating to the project is present, and which has server integration.

[0076] The virtual server (4.2) is an element located within the virtual machine (4) and integrated with the virtual database (4.1 ).

[0077] Git (5) is a version control element that is integrated with the virtual machine (4), from which the project is downloaded to the docker unit (6), and which is used to monitor changes in computer files.

[0078] The docker unit (6) is an element in which the test report generated in the virtual machine (4) is stored and to which the project is downloaded from Git (5).

[0079] The invention enables the dynamic analysis of scenarios at each software change and the recommendation of the most appropriate scenarios. In the impact analysis obtained by analysing the changes between two versions, an analysis process is run to determine the order in which the test scenarios that may be directly and indirectly affected by code changes should be executed in order to achieve the most efficient test process, thereby ensuring the ordering of the selected tests. Thus, the tests required for the smooth execution of test scenarios are also included in the scenario, and a test plan is delivered to the end user via the resulting final ordered list.

[0080] The invention makes it possible for the developed software to perform both user acceptance tests and tests of a desired development package on three different platforms, namely web, iOS, and Android. In this way, it aims both to optimise manual effort through the test ordering flow and to enable the execution of automated tests through a test automation execution flow, ensuring that all tests of the existing software are executed automatically and that the results are reported. Thus, corporate customers requesting the development of the software are enabled to perform user acceptance tests automatically as part of the software life cycle. The invention enables the analysis and prioritisation of test scenarios by using machine learning and artificial intelligence algorithms and demonstrates that a more specialised and focused solution is provided in the field of software test automation.

[0081] The invention enables the provision of solutions that can be directly integrated into software customer acceptance processes through end-user test scenarios, and provides an applicable solution at every stage of software development processes through a dynamic and user-oriented test scenario creation approach, thereby enabling the execution of remote end-to-end testing of software outputs for any profile.

[0082] The software test method comprises the process steps of:

[0083] • Preparation of test scenarios by the test team (100),

[0084] • Uploading of the prepared test scenarios via the interface (110),

[0085] • Analysis of test scenarios by using machine learning and artificial intelligence algorithms (120),

[0086] • Sending of prioritised and reduced test scenarios to the virtual machine (130),

[0087] • Saving of prioritised and reduced test scenarios to the virtual database (140),

[0088] • Execution of testing by the virtual machine (150),

[0089] • Generation of the test report (160),

[0090] • Saving of the report to the docker unit (170),

[0091] • Identification of the requested version in Git by means of machine learning (180),

[0092] • Downloading of the project from Git to the docker unit (190),

[0093] • Performance of project control in the docker unit by means of machine learning for automated test scenarios present in the virtual machine (200),

[0094] • Display of the project downloaded to the docker unit and the saved report on the interface via the server (210).

[0095] Analysing test scenarios by using machine learning and artificial intelligence algorithms (120), which is one of the process steps of the software test method, comprises the process steps of:

[0096] • Scanning of all code changes between two versions marked as a version transition on the version control system (Git, Gerrit, Bitbucket, etc.) (121 ),

[0097] • Indexing of all scanned files in the database (122), • Generation of a dependency map for each file with changes by using dependencies within the file content (123),

[0098] • Merging the dependency maps for all files to combine all changed parts of the project on a single map (124),

[0099] • Completion of impact analysis by scanning all classes on this map via a previously created source code analysis index (125),

[0100] • Determination of test scenarios referenced by the analyses through the identification of analyses that may be affected (126),

[0101] • Combining the determined test scenarios together with mandatory test scenarios and sending them to the test scenario prioritisation module (127).

[0102] Sending prioritised and reduced test scenarios to the virtual machine (130), which is one of the process steps of the software test method, comprises the process steps of:

[0103] • Prioritisation of test scenarios (131 ),

[0104] • Reduction of test scenarios (132).

[0105] Prioritising test scenarios (131 ), which is one of the process steps of sending prioritised and reduced test scenarios to the virtual machine (130) within the software test method, comprises the process steps of:

[0106] • Creation of a document topic matrix by means of a topic modelling algorithm via Docker (131.1 ),

[0107] • Retrieval of the prepared test scenarios (131 .2),

[0108] • Prioritisation of test scenarios by using the document matrix via machine learning (131 .3).

[0109] Reducing test scenarios (132), which is one of the process steps of sending prioritised and reduced test scenarios to the virtual machine (130) within the software test method, comprises the process steps of:

[0110] • Collection of project version information, implemented changes, and fixed defect data via Apache Lucene / Elasticsearch, source code history, and JIRA (132.1 ),

[0111] • Reduction of test scenarios by means of machine learning algorithms using the collected data (132.2). Executing testing by the virtual machine (150), which is one of the process steps of the software test method, comprises the process steps of:

[0112] • Retrieval of the project and modules from the database by the virtual machine (151 ),

[0113] • Retrieval of test scenarios from the virtual database (152),

[0114] • Creation of automatic modules in the virtual machine (153),

[0115] • Retrieval of coverage data from the virtual database (154),

[0116] • Retrieval of project versions from the virtual database (155),

[0117] • Identification of automatic scenarios via the virtual server through the virtual database and the server (156),

[0118] • Retrieval of test scenarios from the virtual database (157),

[0119] • Execution of tests (158).

Claims

CLAIMS1. A server-based software test platform, comprising;• an interface (2) that enables interaction of the system with the user and the display of the project and reports,• a database (1 ) in which the project is stored and which operates in an integrated manner with the virtual machine (4),• a server (3) operating as an intermediary between the interface and the docker unit (6),• a virtual machine (4) that enables the execution of tests,• a virtual database (4.1 ) located within the virtual machine (4) and in which reduced and prioritised scenarios are stored,• a virtual server (4.2) located within the virtual machine (4) and operating in an integrated manner with the virtual database (4.1 ),• Git (5), which is a version control element integrated with the virtual machine (4), from which the project is downloaded to the docker unit (6) and which is used to monitor changes in computer files, and• a docker unit (6) in which the report is stored and to which the project is downloaded from Git (5).

2. A software test method, comprising the process steps of;• preparation of test scenarios by the test team (100),• uploading of the prepared test scenarios via the interface (110),• analysing test scenarios by using machine learning and artificial intelligence algorithms (120),• sending prioritised and reduced test scenarios to the virtual machine (130),• saving prioritised and reduced test scenarios to the virtual database (140),• executing testing by the virtual machine (150),• generating the test report (160),• saving the report to the docker unit (170),• identifying the requested version in Git by means of machine learning (180),• downloading the project from Git to the docker unit (190),• performing project control in the docker unit by means of machine learning for automated test scenarios present in the virtual machine (200), and• displaying the project downloaded to the docker unit and the saved report on the interface via the server (210).

3. The software test method according to Claim 2, wherein the process step of analysing test scenarios by using machine learning and artificial intelligence algorithms (120) comprises the process steps of;• scanning all code changes between two versions marked as a version transition on the version control system (121 ),• indexing all scanned files in the database (122),• generating a dependency map for each file with changes by using dependencies within the file content (123),• merging the dependency maps for all files to combine all changed parts of the project on a single map (124),• completing impact analysis by scanning all classes on this map via a previously created source code analysis index (125),• determining test scenarios referenced by the analyses through the identification of analyses that may be affected (126), and• combining the determined test scenarios together with mandatory test scenarios and sending them to the test scenario prioritisation module (127).

4. The software test method according to Claim 3, wherein the process step of scanning all code changes between two versions marked as a version transition on the version control system (121 ) comprises version control systems such as Git, Gerrit, and Bitbucket.

5. The software test method according to Claim 2, wherein the process step of sending prioritised and reduced test scenarios to the virtual machine (130) comprises the process steps of;• prioritising test scenarios (131 ), and• reduction of test scenarios (132).

6. The software test method according to Claim 2 or Claim 3, wherein the process step of prioritising test scenarios (131 ) comprises the process steps of;• creating a document topic matrix by means of a topic modelling algorithm via Docker (131.1 ),• retrieving the prepared test scenarios (131 .2), and• prioritising test scenarios by using the document matrix by means of machine learning (131 .3).

7. The software test method according to Claim 2 or Claim 3, wherein the process step of reduction of test scenarios (132) comprises the process steps of;• collecting project version information, implemented changes, and fixed defect data via Apache Lucene / Elasticsearch, source code history, and JIRA (132.1 ), and• reducing test scenarios by means of machine learning algorithms using the collected data (132.2).

8. The software test method according to Claim 2, wherein the process step of executing testing by the virtual machine (150) comprises the process steps of;• retrieving the project and modules from the database by the virtual machine (151 ),• retrieving test scenarios from the virtual database (152),• creating automatic modules in the virtual machine (153),• retrieving coverage data from the virtual database (154),• retrieving project versions from the virtual database (155),• identifying automatic scenarios via the virtual server through the virtual database and the server (156),• retrieving test scenarios from the virtual database (157),• executing tests (158).

Images