Automated generation of software application test cases for evaluating software application issues

Abstract

Techniques are provided for automated generation of software application test cases for evaluating software application issues. A method includes obtaining a first mapping of log event templates related to log events in a software application log to respective log event template vectors; obtaining a second mapping of test step vectors generated using the log event template vectors to respective test step functions, wherein a given test step vector comprises one or more of the log event template vectors; in response to obtaining the information characterizing the software application problem: generating a test step vector representation of the information characterizing the software application problem using the first mapping; mapping the test step vector representation of the information to a corresponding test step function using a second mapping; and generating a test case logical flow using the mapped test step function to evaluate the software application problem.

Description

Background Art

[0001] It is often necessary to generate test cases to evaluate reported issues related to the operation of a given software product. However, traditional methods for evaluating reported software issues often include generating test cases using manual techniques that are time consuming and prone to errors. Furthermore, such manual techniques may result in test cases that do not properly evaluate the operation of the software product, which may result in reduced software quality. Summary of the invention

[0002] The illustrative implementation of the present disclosure provides a technique for automatically generating software application test cases for evaluating software application problems (e.g., vulnerabilities and errors). An exemplary method includes obtaining a first mapping of a plurality of log event templates related to one or more log events in one or more software logs generated by executing a software application on one or more of a plurality of information technology assets of an information technology infrastructure to corresponding vector representations in a vector representation of the log event templates; obtaining a second mapping of a plurality of test step vector representations generated using the vector representation of the log event templates to corresponding test step functions in a plurality of test step functions, wherein a given test step vector representation includes one or more vector representations in the vector representation of the log event templates, and wherein the second mapping is generated by analyzing the execution of a plurality of test steps related to the software application in the execution history of the one or more software logs; in response to obtaining information characterizing a software problem related to the software application: generating one or more test step vector representations of the information characterizing the software problem using the first mapping; mapping the one or more test step vector representations of the information characterizing the software problem to corresponding test step functions in a plurality of test step functions using the second mapping; and automatically generating a test case logic flow using the mapped test step functions to evaluate the software problem related to the software application.

[0003] The illustrative implementations may provide significant advantages over conventional techniques for evaluating software applications. For example, in one or more implementations, one or more software application test cases for a given software application are automatically generated by analyzing one or more software logs associated with historical execution of the given software application, overcoming problems associated with time-consuming and error-prone manual software evaluation techniques.

[0004] These and other illustrative implementations include, but are not limited to, methods, apparatus, networks, systems, and processor-readable storage media. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 is a block diagram of an information processing system configured, in an illustrative implementation, for automatically generating software application test cases for evaluating software application problems;

[0006] Figure 2 A text vectorization technique using bag-of-word natural language processing in an illustrative implementation is shown;

[0007] Figure 3 shows examples of different product types having different associated test log formats in an illustrative implementation;

[0008] Figure 4 A log event template dictionary used in an illustrative implementation to convert a log event into a log event template is shown;

[0009] Figure 5 Automated generation of software application test cases for evaluating software application problems in an illustrative implementation is shown;

[0010] Figure 6 An exemplary log event template dictionary in an illustrative implementation is shown;

[0011] Figure 7 An exemplary test step vector to test step function mapping table in an illustrative implementation is shown;

[0012] Figure 8 The processing of one or more raw software logs in an illustrative implementation to generate Figure 7 The test step vector to test step function mapping table;

[0013] Fig. 9 shows the processing of one or more client software logs in an illustrative implementation to generate a test case logic flow that evaluates reported software issues associated with a given software application;

[0014] Fig.10 shows an exemplary customer test step vector to test step function mapping table in an illustrative implementation;

[0015] Fig.11 is a flow chart illustrating an exemplary implementation of a process for automating generation of software application test cases for evaluating software application problems according to an implementation; and

[0016] Fig.12 and Fig.13 An example of a processing platform that may be used to implement at least a portion of an information handling system in an illustrative implementation is shown. DETAILED DESCRIPTION

[0017] Illustrative implementations will be described herein with reference to an exemplary information handling system and associated computers, servers, storage devices, and other processing devices. However, it should be understood that the implementations are not limited to use with the specific illustrative system and device configurations shown. Therefore, the term "information handling system" as used herein is intended to be interpreted broadly so as to cover, for example, processing systems including cloud computing and storage systems, as well as other types of processing systems including various combinations of physical and virtual processing resources. Thus, an information handling system may include, for example, at least one data center or other type of cloud-based system including one or more clouds that host tenants that access cloud resources.

[0018] As mentioned above, test cases are often generated to evaluate reported issues related to the operation of a software product. However, when customers or other non-technical users of a given software product encounter and report issues, it is often difficult to reproduce such issues. For example, the customer's report may not be clear enough to provide a clear understanding of the software issue. The customer may not provide enough information (e.g., steps to reproduce a given software issue, execution environment configuration, screenshots, or relevant error messages) to reproduce a given software vulnerability or other issue. This can make it difficult for the testing team to understand the problem and reproduce it in a controlled environment.

[0019] Figure 1 An information processing system 100 configured according to an illustrative implementation is shown. It is assumed that the information processing system 100 is built on at least one processing platform and provides functionality for automatically generating software application test cases for evaluating software application problems. The information processing system 100 includes a set of client devices 102-1, 102-2, ... 102-M (collectively referred to as client devices 102) coupled to a network 104. An IT infrastructure 105 is also coupled to the network 104, and the IT infrastructure includes one or more IT assets 106, a test database 108, and a software application test case generation system 110. The IT assets 106 may include physical and / or virtual computing resources in the IT infrastructure 105. Physical computing resources may include physical hardware, such as servers, host devices, storage systems, networking equipment, Internet of Things (IoT) devices, other types of processing and computing devices, including desktops, laptops, tablet computers, smart phones, etc. Virtual computing resources may include virtual machines (VMs), containers, etc.

[0020] The IT assets 106 of the IT infrastructure 105 can host software applications utilized by corresponding client devices in the client devices 102, such as according to a client-server computer program architecture. In some implementations, the software applications include network applications designed to be delivered from assets in the IT infrastructure 105 to users (e.g., users of the client devices 102) via the network 104. Various other examples are possible, such as situations where one or more software applications are used internally in the IT infrastructure 105 and are not exposed to the client devices 102. It should be understood that in some implementations, some of the IT assets 106 of the IT infrastructure 105 can themselves be considered as applications to be evaluated, or more generally software or hardware. For example, individual IT assets in the IT assets 106 that are virtual computing resources implemented as software containers can represent software to be evaluated. As another example, individual IT assets in the IT assets 106 that are physical computing resources can represent hardware devices to be evaluated.

[0021] The software application test case generation system 110 utilizes various information stored in the test database 108, such as an execution log that provides information obtained from executing a given software application, to automatically generate test cases for the software application to reproduce a given software application problem. In some implementations, the software application test case generation system 110 is used for an enterprise system. For example, an enterprise can subscribe to or otherwise utilize the software application test case generation system 110 to automatically generate software application test cases to reproduce software application problems. As used herein, the term "enterprise system" is intended to be interpreted broadly to include any set of systems or other computing devices. For example, the IT assets 106 of the IT infrastructure 105 can provide part of one or more enterprise systems. A given enterprise system may also or alternatively include one or more devices in the client device 102. In some implementations, the enterprise system includes one or more data centers, a cloud infrastructure (which includes one or more clouds), etc. A given enterprise system, such as a cloud infrastructure, can host assets associated with multiple enterprises (e.g., two or more different enterprises, organizations, or other entities).

[0022] Client devices 102 may include, for example, any combination of physical computing devices, such as IoT devices, mobile phones, laptop computers, tablet computers, desktop computers, or other types of devices utilized by enterprise members. Such devices are examples of devices more generally referred to herein as "processing devices." Some of these processing devices are also generally referred to herein as "computers." Client devices 102 may also or alternatively include virtual computing resources, such as VMs, containers, and the like.

[0023] In some implementations, client devices 102 include respective computers associated with a particular company, organization, or other enterprise. Thus, client devices 102 may be considered examples of assets of an enterprise system. Additionally, at least portions of information processing system 100 may also be referred to herein as collectively comprising one or more "enterprises." Those skilled in the art will appreciate that many other operational scenarios involving various different types and arrangements of processing nodes are possible.

[0024] Network 104 is assumed to include a global computer network such as the Internet, although other types of networks may be part of network 104, including a wide area network (WAN), a local area network (LAN), a satellite network, a telephone or wired network, a cellular network, a wireless network such as a WiFi or WiMAX network, or various portions or combinations of these and other types of networks.

[0025] As discussed above, the test database 108 is configured to store and record various information, such as an execution log that provides information obtained from executing a given software application, which is used by the software application test case generation system 110 to automatically generate software application test cases to reproduce a given software application problem. Such information may include, but is not limited to, information related to the execution of one or more software applications, test cases, test targets, test points, test coverage, test plans, etc. In some implementations, the test database 108 is implemented using one or more storage systems or devices associated with the software application test case generation system 110. In some implementations, one or more of the storage systems used to implement the test database 108 include a scaled-out all-flash content addressable storage array or other type of storage array.

[0026] Therefore, the term "storage system" as used herein is intended to be interpreted broadly and should not be considered limited to content addressable storage systems or flash-based storage systems. A given storage system as a broad term used herein may include, for example, network attached storage (NAS), storage area network (SAN), direct attached storage (DAS), and distributed DAS, as well as combinations of these and other storage types, including software-defined storage.

[0027] In the illustrative implementation, other specific types of storage products that can be used to implement the storage system include all flash and hybrid flash arrays, software-defined storage products, cloud storage products, object-based storage products, and extended NAS clusters. In the illustrative implementation, a combination of multiple storage products in these and other storage products can also be used to implement a given storage system.

[0028] Despite Figure 1Although not explicitly shown, one or more input-output devices such as a keyboard, display, or other types of input-output devices may be used to support one or more user interfaces to the software application test case generation system 110, as well as to support communications between the software application test case generation system 110 and other related systems and devices not explicitly shown.

[0029] The client device 102 is configured to access or otherwise utilize the IT infrastructure 105. In some implementations, it is assumed that the client device 102 is associated with a user who executes one or more software applications and reports vulnerabilities or other problems encountered in such execution. In other implementations, it is assumed that the client device 102 is associated with a system administrator, IT manager, or other authorized person responsible for managing the IT assets 106 of the IT infrastructure 105 (e.g., where such management includes performing tests on the IT assets 106 or applications or other software running on the IT assets 106). For example, a user can operate a given one of the client devices 102 to access a graphical user interface (GUI) provided by the software application test case generation system 110 to manage test plans (e.g., create, review, execute, etc.). The software application test case generation system 110 can be provided as a cloud service that can be accessed by a given client device 102 to allow its user to manage test plans. In some implementations, the IT assets 106 of the IT infrastructure 105 are owned or operated by the same enterprise that operates the software application test case generation system 110 (e.g., where an enterprise such as a business provides support for its operated assets). In other implementations, the IT assets 106 of the IT infrastructure 105 may be owned or operated by one or more enterprises different from the enterprise operating the software application test case generation system 110 (e.g., a first enterprise provides support for assets owned by multiple different customers, businesses, etc.) Various other examples are possible.

[0030] In other implementations, instead of providing support for IT assets 106 of IT infrastructure 105, or in addition to providing support for the IT assets, the software application test case generation system 110 can provide support for testing of client devices 102. For example, the software application test case generation system 110 can be operated by a hardware vendor that manufactures and sells computing devices (e.g., desktops, laptops, tablet computers, smart phones, etc.), and the client device 102 represents a computing device sold by the hardware vendor. The software application test case generation system 110 can also or alternatively be operated by a software vendor that produces and sells software (e.g., applications) running on the client device 102. However, the software application test case generation system 110 does not need to be operated by any single hardware or software vendor. Instead, the software application test case generation system 110 can be provided as a service to provide support for computing devices or software sold by any number of hardware or software vendors. The client device 102 can subscribe to the software application test case generation system 110 to provide support for testing and / or evaluation of the client device 102 or the software running thereon. Various other examples are possible.

[0031] In some implementations, the client device 102 may implement a host agent configured to automatically transmit information about the state of the client device 102 (e.g., such as in the form of test and / or execution logs that are periodically provided to the test database 108 and / or the software application test case generation system 110). Such a host agent may also or alternatively be configured to automatically receive commands from the software application test case generation system 110 to perform remote actions (e.g., run various test steps and / or test cases on the client device 102 and / or the IT assets 106 of the IT infrastructure 105). The host agent may similarly be deployed on the IT assets 106 of the IT infrastructure 105.

[0032] It should be noted that a "host agent" as that term is generally used herein may include an automated entity, such as a software entity running on a processing device. Thus, a host agent need not be a human entity.

[0033] Assume that at least one processing device is used to implement Figure 1 The software application test case generation system 110 is implemented. Each such processing device generally includes at least one processor and associated memory, and implements one or more functional modules or logic for controlling certain features of the software application test case generation system 110. Figure 1In implementation, the software application test case generation system 110 includes a log vectorization module 112, a test step log vector to test step function mapper 114, and an automated test case logic generator 116. The log vectorization module 112 is configured to obtain a set of software logs generated by executing one or more software applications on one or more IT assets 106 of, for example, the IT infrastructure 105 or on a client device 102. The log vectorization module 112 is also configured to parse the set of software logs to generate a set of log event templates for testing actions performed during execution of the one or more software applications on the one or more IT assets 106 or the client device 102. The log vectorization module 112 is further configured to generate vector representations of a plurality of log events using the generated set of log event templates. The test step log vector to test step function mapper 114 is configured to perform mapping of one or more test step vector representations of software issues to corresponding test step functions in the test step function using one or more machine learning based algorithms. The automated test case logic generator 116 is configured to generate a test case logic flow using the mapped test step functions to evaluate software issues related to a given software application. The software application test case generation system 110 is further configured to execute the test case logic flow in one or more IT assets in the IT assets 106 of the IT infrastructure 105 directly or via one or more of the client devices 102.

[0034] It should be understood that Figure 1 The specific arrangements of the client device 102, IT infrastructure 105, and software application test case generation system 110 shown in the implementation are presented by way of example only, and alternative arrangements may be used in other implementations. As discussed above, the software application test case generation system 110 (or portions of its components, such as one or more of the log vectorization module 112, the test step log vector to test step function mapper 114, and the automated test case logic generator 116) may be implemented internally in one or more of the client device 102 and / or the IT infrastructure 105 in some implementations.

[0035] At least portions of the log vectorization module 112 , the test step log vector to test step function mapper 114 , and the automated test case logic generator 116 may be implemented at least in part in the form of software stored in a memory and executed by a processor.

[0036] The software application test case generation system 110 and other parts of the information processing system 100 (described in further detail below) may be part of a cloud infrastructure.

[0037] Assumptions Figure 1The software application test case generation system 110 and other components of the information processing system 100 in the implementation are implemented using at least one processing platform, which includes one or more processing devices, each processing device having a processor coupled to a memory. Such processing devices can illustratively include a specific arrangement of computing, storage, and network resources.

[0038] The client devices 102, IT infrastructure 105, test database 108, and software application test case generation system 110 or components thereof (e.g., log vectorization module 112, test step log vector to test step function mapper 114, and automated test case logic generator 116) may be implemented on respective different processing platforms, although many other arrangements are possible. For example, in some implementations, at least a portion of the software application test case generation system 110 and one or more of the client devices 102, IT infrastructure 105, and / or test database 108 are implemented on the same processing platform. Thus, a given client device (e.g., client device 102-1) may be implemented at least in part within at least one processing platform that implements at least a portion of the software application test case generation system 110.

[0039] The term "processing platform" as used herein is intended to be interpreted broadly so as to include, by way of illustration and not limitation, multiple groups of processing devices and associated storage systems configured to communicate over one or more networks. For example, a distributed implementation of the information processing system 100 is possible, in which certain components of the system reside in a data center in a first geographic location, while other components of the system reside in one or more other data centers in one or more other geographic locations that may be remote from the first geographic location. Therefore, in some implementations of the information processing system 100, the client device 102, IT infrastructure 105, IT assets 106, test database 108, and software application test case generation system 110, or portions or components thereof, may reside in different data centers. Many other distributed implementations are possible. The software application test case generation system 110 may also be implemented in a distributed manner across multiple data centers.

[0040] The following will be combined Fig.12 and Fig.13 Additional examples of processing platforms used to implement software application test case generation system 110 and other components of information processing system 100 in illustrative implementations are described in more detail.

[0041] It should be understood that these and other features of the illustrative implementations are presented by way of example only and should not be construed as limiting in any way.

[0042] It should be understood that Figure 1The set of specific elements shown for automatically generating software application test cases for evaluating software application problems is presented only by way of illustrative examples, and in other implementations, additional or alternative elements may be used. Thus, another implementation may include additional or alternative systems, devices, and other network entities, as well as different arrangements of modules and other components.

[0043] It should be understood that these and other features of the illustrative implementations are presented by way of example only and should not be construed as limiting in any way.

[0044] Illustrative implementations provide techniques for automatically generating software application test cases to evaluate software application problems. In some implementations, the disclosed software application test case generation techniques are based at least in part on analysis of system test logs (also referred to as "software logs") to improve the evaluation of software application problems.

[0045] Software log vectorization will now be described. Various NLP methods can be used for text vectorization, including bag of words, word2vec, etc. The text vectorization model can create an index for each word and then use such index to represent the sentence. For example, Figure 2 A process for bag-of-words text vectorization is shown, in which a dictionary 201 is generated from a set of sentences 203, which are converted into text vector sets 205-1, 205-2, and 205-3. The text vector 205-1 has the problem that different sentences have different vector dimensions, which is not suitable for input to a machine learning algorithm. Therefore, the text vector 205-1 can be transformed into a text vector 205-2, which includes a 6-dimensional vector for each sentence in the sentence (for example, using each word frequency in a sentence instead of a word index). This is used for bag-of-words text vectorization. Word2vec can be used because bag-of-words text vectorization cannot reflect word relationships, and the human word corpus is huge, which causes the vector dimension to be too large and cannot be calculated in any algorithm. In a system evaluation environment, software logs are semi-structured and have limited word descriptions. Time series are also important for software logs. In some implementations, a modified bag-of-words text vectorization method is used, which eliminates its shortcomings and makes it suitable for application in log-based test case generation scenarios.

[0046] Individual words in software logs are often not always meaningful (e.g., they are not in human-readable form). A log statement (e.g., a line or entry in a software log) can be considered a log event. Software log features depend not only on individual log events, but also on log event sequences, frequencies, inter-arrival times (e.g., average inter-arrival times), time interval distributions, etc. Traditional log vectorization models cannot avoid coordinate transformations (e.g., from words to logs, from logs to sequences) and also have high computational costs (e.g., for training), which may be prohibitive for large-scale testing environments. However, conventional log vectorization models that can be designed for log anomaly detection can abstract features in a different way than those used for test case generation.

[0047] Different software logs (e.g., for different products) may have their own formats and templates. For example, Figure 3 Two product types 301-A and 301-B are shown with different associated software log formats 303-A and 303-B. For example, product type 301-A may be an Internet server with an associated software log format 303-A focused on information transformation. Product type 301-B may be a storage-related product with an associated software log format 303-B focused on logical events. Various other examples are possible. Depending on the log structure of a particular software log, it is important to identify the log format and associated log template for that particular software log for pre-processing.

[0048] In some implementations, the log vectorization process extracts the constant portion from the log entry. Consider the following log entry as an example:

[0049] "A_QA_ACTION[DEACTIVATE:Appliance-WX-D0579-node-A-PM]Finished-Deactivating Appliance-WX-D0579-node-A-PM:pid=16752uptime=1 / 11 / 2021 13:33:52DeactivateSig=6",

[0050] By extracting the constant part, it is structured into the following log event template:

[0051] "*_QA_ACTION[DEACTIVATE:*-PM]Finished-Deactivating Appliance-*-PM:pid=*uptime=*DeactivateSig=6".

[0052] The software log can be transformed into a combination of several log event templates, where the general principle of the log event template is to ignore variables (e.g., numbers, object values, etc.) while retaining the logic and other parts of the log event (e.g., constant parts). The process of parsing the software log to generate a log event template can be expressed as follows:

[0053]

[0054] Where l represents a line in the original log message, N represents the total number of lines in the original log, and l i represents the i-th line of the original log, where 1≤i≤N, A represents a function for passing each line to a log event template, as described above, ET represents a log event template, and LT represents a log template consisting of a set of log event templates.

[0055] In order to reproduce a reported software problem or vulnerability, it is often important to identify the test steps that will trigger the original problem. From a logging perspective, one test step or action maps to a block of log messages, which includes a detailed command and related product reaction sequence. Therefore, in at least some implementations, vectorization of these blocks of log messages is useful for identifying appropriate test steps to reproduce a reported software problem.

[0056] A given software log may be divided into one or more blocks including a number of log steps (e.g., log events). A given software log may be divided by identifying keywords such as "actions". Such keywords may be determined as part of the design phase of a given software application. Assume that there are m test step logs in a given log, and the window size may be expressed as follows:

[0057] W=[W 1 ,W 2 ,W 3 ,…W m ],

[0058] The window size determines the length of a single test step log (consisting of multiple lines of log data or log events). In a given software log, there are m log blocks, each of which corresponds to a specific test step log. The i-th test step log l si Including the following W i Log message:

[0059]

[0060] Each log message in a given test step log can be parsed into a log event template. Therefore, a test step log can be parsed into a list of log event templates. The position of the log event in the larger list provides an indication of the sequence of log events. si It can be expressed as follows:

[0061]

[0062] Where l represents a line of the original log message, l si represents the i-th test step log, where 1≤i≤m, W i represents the log window size of the i-th test step log, W represents the window size set of a test log, m represents the number of test steps in a test log, A is an abstract function used to pass each line in the software log to the log event template, as mentioned above, ET represents the log event template, and LST i Represents the log template for the i-th test step.

[0063] The process for log vector generation may include creating a log event template dictionary (1≤i≤Z), where Z is the number of unique log event templates, and translating the log event template using the log event template dictionary denoted as D, such as Figure 4 The log event template dictionary 400 is shown. Figure 4 An example of a log event template 405 converted using a dictionary and function D is also shown. The different lengths of the log event template 405 reflect the different lengths of the original log message. The maximum length of the original log message can be determined according to the following:

[0064] X=Maximum(W k )(1≤k≤z).

[0065] Log event templates that are shorter than the maximum length can be padded with zero values, so the dictionary can have zero elements added, such as Figure 4 As shown in Table 410. Each log template can be vectorized according to the following content:

[0066]

[0067] Among them LST k represents the kth test step log template, 1≤k≤m; V(LST k ) represents the vector representation of the log template of the kth test step, and V(LST k ) belongs to the test step log vector space T; Z indicates the length of the unique log event template group, and X represents the total number of lines of the longest test step log template. D represents a function for translating a log event template into a vector using the created log event template dictionary 400. ET represents a log event template, and i represents the i-th log event template. Where 1≤i≤Z. W krepresents the total number of lines of the kth test step log template. A test step log vector can naturally show a sequence of test events, and the dimension of the log vector is X, which should not be a huge uncomputed number. The dictionary capacity can also be customized so that it is acceptable in different product domains. For example, if the longest test step log template has a length of 16 lines, then X is equal to 16. If a given test step has a length of five lines, then k , then W k Satisfy W k <X, and use the second expression above, inserting 11 ((XW k )=(16-5))zero, as shown below:

[0068] [3,2,2,7,1,0,0,0,0,0,0,0,0,0,0,0],

[0069] So that the vector representations of the test step log templates will have the same length.

[0070] Figure 5 The overall process flow 500 for automatically generating software application test cases for evaluating software application problems in an illustrative implementation is shown. The process flow 500 begins with a raw software log set 501 pre-processed in a log vectorization phase 503 and a test step log vector to test step function mapping generation phase 505. In the log vectorization phase 503, a log event template dictionary 530 (e.g., in the form of Figure 4 The log event template dictionary 400 is used to parse the original software log 501 to identify the log event template set 532 (for example, as described above in conjunction with Figure 5 The process flow 500 then proceeds to the test step log vector to test step function mapping generation phase 505, which processes the vector representation of the test step log template. Sometimes referred to herein as test step vectors, and a test step vector to test step function mapping table 552 is generated that maps such test step vectors to test step automation functions TSF. In at least some implementations, the test step vector to test step function mapping table 552 is generated by performing regression analysis on historical test case execution logs, for example, as described below in conjunction with Figure 7 The test step vector to test step function mapping table 552 is sometimes referred to herein as a mapping function M and is represented as follows:

[0071]

[0072] Where TSF refers to the given test step log template LST K The test step automation function is expressed as

[0073] In the real-time processing phase, e.g., in response to a customer reporting an issue with a given software product (e.g., one or more software errors and / or vulnerabilities), the process flow 500 includes an automated test case logic generation phase 507. In the automated test case logic generation phase 507, one or more customer software logs 510 are processed by a customer software log analysis module 570 that obtains test step log vectors from the customer software logs in a manner similar to that in the pre-processing phase described above (e.g., because customer software logs for a given software product are typically substantially similar to the original software logs 501 associated with the given software product), as follows:

[0074]

[0075] In addition, similarly, the test step log vector generator 572 uses, for example, an approximate nearest neighbor method (such as Annoy) to identify a given test step vector V (LST i The first N (e.g., N=10) groups of similar vectors of the log template of the i-th test step are represented as Annoy(V(LST i ))=[v 1 ,v 2 ,v 3 ,…v 10 ]. The first 10 similar vectors in the vector space T comprise the test step function candidate TSF C , and can be far away from the target vector (e.g., customer problem log vector). Test Step Function Candidate TSF C It can be expressed as follows:

[0076] TSF C =[M(v 1 ),M(v 2 ),M(v 3 ),…M(v 10 )],

[0077] The mapping function M is based on a test step vector to test step function mapping table 552 .

[0078] In some implementations, the test step function candidate TSF C can be evaluated by a test step function candidate evaluator 574, which generates a test step function candidate TSF from the test step function candidate C Select a given test step function, e.g., based on the test step function candidate TSF C , with the highest occurrence count (e.g., in the test step function candidate TSF CThe most common test step function candidates in TSF C ),as follows:

[0079]

[0080] In the output list, each TSF candidate number is counted, and the candidate TSF with the largest count can be marked as the target TSF (e.g., if the logs are similar, the corresponding TSFs should be the same). However, if the counts of all TSFs are 1, the top N similar vectors are far from the target vector, and the new TSF is added to the test step vector to test step function mapping table 552.

[0081] The test case logic step generator 576 aggregates the selected TSF for each test step log template identified in the client software log to generate a set of test case logic steps, as follows:

[0082]

[0083] The logical step set T of the test case can be executed C (L TC ) to reproduce the identified customer problem. In some implementations, the software logs are also processed to extract the hardware configuration information for the customer environment, and the set of test case logic steps T C (L TC ) can be executed on one or more most similar hardware devices.

[0084] Figure 6 An exemplary log event template dictionary 600 in an illustrative implementation is shown. Figure 6 In the example of FIG. 6 , the log event template dictionary 600 includes 20 unique log event templates. The log event template dictionary 600 can be used with Figure 4 The log event template dictionary 400 is created in a similar manner.

[0085] Figure 7 An exemplary test step vector to test step function mapping table 700 in an illustrative implementation is shown. Figure 7 In the example of , a test step vector to test step function mapping table 700 is generated by performing regression analysis on the historical test case execution log to identify the test step vectors associated with the test step function. As described above, the test step vector to test step function mapping table 700 is sometimes referred to as mapping function M in this article. The test step vector to test step function mapping table 700 indicates that the analysis of the historical test case execution log identified 14 test step vectors (LST 1 To LST 14 ), each test step vector consists of seven log event templates (ET 1to ET 7 ) to describe a test step.

[0086] The log event template dictionary 600 and the test step vector to test step function mapping table 700 are further discussed below in conjunction with exemplary generation of software application test cases to evaluate a given software application problem.

[0087] It is important to note that, for example, the same test step function will generate similar (though not identical) test logs due to differences in the execution environment. Figure 7 The test step vector (V(LST 1 )、V(LST 2 )、V(LST 3 )) are mapped to TSF 1 .

[0088] Figure 8 In an illustrative implementation, one or more raw software logs 805 are processed to generate Figure 7 The test step vector to test step function mapping table. Figure 8 In the example of FIG. 8 , one or more raw software logs 805 are processed by a log vectorization process 810 , for example, using Figure 6 The log event template dictionary 600 is used to generate the corresponding log event template 815 [e.g., ET 1 ,ET 2 ,ET 4 ,ET 6 ,ET 19 ,ET 2 ,…,ET 4 ,ET 3 The test step vector generation process 820 processes the log event template 815 to generate a corresponding test step vector (V(LST)) by grouping one or more individual log event templates into test steps. 1 ),V(LST 2 ),…V(LST N )), as described above.

[0089] The test step vector generated by the test step vector generation process 820 is then translated into a test step function 840 using the test step vector to test step function mapping process 830, as described above in conjunction with Figure 7 discussed.

[0090] Fig. 9The processing of one or more customer software logs 905 (e.g., logs generated from the execution of a given software application in a customer environment) in an illustrative implementation to generate a test case logic flow that evaluates reported software issues related to the given software application is shown. Fig. 9 In the example of FIG. 1 , one or more client software logs 905 are processed by a client log vectorization process 910, for example, by grouping one or more individual log event templates into test steps using Figure 6 The log event template dictionary 600 and Figure 8 The test step vector generation process 820 is used to generate the corresponding customer test step vector (V(CLST 1 ),V(CLST 2 ),…V(CLST 3 )…), as discussed above.

[0091] In addition, similar test step log vector generation process 930 uses, for example, the Annoy approximate nearest neighbor method to identify a given customer test step vector V (CLST i ) (e.g., the vector representation of the i-th customer test step log template) of the first N (e.g., N=5) groups of similar vectors, denoted as Annoy V (CLST i )=[v 1 ,v 2 ,v 3 ,…v 5 ]. The first five similar vectors in the vector space T comprise the test step function candidate TSF C , and may be far from the target vector (e.g., customer problem log vector). Test Step Function Candidate TSF C It can be expressed as follows:

[0092] TSF C =[M(v 1 ),M(v 2 ),M(v 3 ),…M(v 10 )].

[0093] exist Fig. 9 In the example of , five test step log vectors (V(CLST 1 ),V(CLST 2 ),V(CLST 3)…). Thereafter, the test step function mapping process 940 is applied to the five similar test step log vectors for each of the customer test step vectors in the customer test step vectors. The test step function selection and test case logic step generation process 950 selects a given test step function from the test step function candidates generated by the test step function mapping process 940, for example, based on the test step function candidate TSF with the highest occurrence count. C , as discussed above. The first three customer test step vectors (V(CLST 1 ),V(CLST 2 ),V(CLST 3 )…)’s similarity, test step function mapping, and test step function selection can be expressed as:

[0094] V(CLST 1 ) :

[0095] (V(CLST 1 ))=print

[0096] (AnnoyIndex(V(CLST 1 )).get_nns_by_item(Index(V(CLST 1 )),5))=[V(CLST 1 ),V(LST 1 ),V(LST 3 ),V(LST 5 ),V(LST 4 ),V(LST 2 )]

[0097] V(CLST 1 )'s test step function mapping :

[0098] M[V(CLST 1 ),V(LST 1 ),V(LST 3 ),V(LST 5 ),V(LST 4 ),V(LST 2 )]=[TSF 1 ,TSF 1 ,TSF 3 ,TSF 2 ,TSF 1 ]

[0099] V (C LST 1 )'s test step function selection:

[0100]

[0101] V(CLST 2 ) :

[0102] (V(CLST 2 ))=[V(CLST 2 ),V(LST 11 ),V(LST 13 ),V(LST 10 ),V(LST 14 ),V(LST 12 )]

[0103] V(CLST 2 )'s test step function mapping :

[0104] M[V(CLST 2 ),V(LST 11 ),V(LST 13 ),V(LST 10 ),V(LST 14 ),V(LST 12 )]=[TSF 6 ,TSF 6 ,TSF 6 ,TSF 6 ,TSF 7 ]

[0105] V(CLST 2 )'s test step function selection:

[0106]

[0107] V(CLST 3 ) :

[0108] (V(CLST 3 ))=[V(CLST 2 ),V(LST 6 ),V(LST 7 ),V(LST 4 ),V(LST 5 ),V(LST 3 )]

[0109] V(CLST 3 )'s test step function mapping :

[0110] M[V(CLST 2 ),V(LST 7 ),V(LST 6 ),V(LST 4 ),V(LST 5 ),V(LST 2 )]=[TSF 2 ,TSF 4 ,TSF 2 ,TSF3 ,TSF 1 ]

[0111] V(CLST 3 )'s test step function selection:

[0112]

[0113] As described above, the test step function selection and test case logic step generation process 950 selects a given test step function from the test step function candidates generated by the test step function mapping process 940. In addition, the test step function selection and test case logic step generation process 950 also generates a test step function for each customer test step vector (V(CLST i )) Collect the selected test step functions as a test case logic step set, as follows:

[0114]

[0115] The generated test case logical step set (T C (L TC )) to evaluate reported customer software issues.

[0116] Fig.10 An exemplary customer test step vector to test step function mapping table 1000 in an illustrative implementation is shown. Fig.10 In the example of FIG. 1 , the customer test step vector to test step function mapping table 1000 reflects the customer test step vector (V(CLST i ))'s test step function selection, as above combined with Fig. 9 The customer test step vector to test step function mapping table 1000 indicates that the analysis of the test case execution log identifies three customer test step vectors (LST 1 To LST 3 ), each customer test step vector consists of seven log event templates (ET 1 to ET 7 ) to describe a test step. Fig.10 The test step function [TSF 1 ,TSF 6 ,TSF 2 ]Depend on Fig. 9 The test step function selection and test case logic step generation process 950 is generated.

[0117] Fig.11 is a flow chart illustrating an exemplary implementation of a process for automatically generating software application test cases for evaluating software application problems according to an implementation. Fig.11In the example, in step 1102, a first mapping of multiple log event templates related to one or more log events in one or more software logs to corresponding vector representations in a vector representation of the log event templates is obtained, and the multiple log event templates are generated by executing a software application on one or more of a plurality of information technology assets of an information technology infrastructure.

[0118] In step 1104, a second mapping of multiple test step vector representations generated using the vector representation of the log event template to corresponding test step functions in the multiple test step functions is obtained, wherein a given test step vector representation includes one or more vector representations in the vector representation of the log event template, and wherein the second mapping is generated by analyzing the execution of multiple test steps related to the software application in the execution history of one or more software logs.

[0119] In response to obtaining information characterizing a software problem associated with a software application in step 1106, one or more test step vector representations of the information characterizing the software problem are generated using a first mapping in step 1108. In step 1110, one or more test step vector representations of the information characterizing the software problem are mapped to corresponding test step functions in a plurality of test step functions using a second mapping. In step 1112, a test case logic flow is automatically generated to evaluate the software problem associated with the software application using the mapped test step functions.

[0120] In some implementations, a log event template dictionary (e.g. Figure 4 The first mapping is implemented by generating a log event template dictionary 400 of the software log event template dictionary, the log event template dictionary being generated by: obtaining a software log generated by executing a software application on an information technology asset; parsing the software log to generate a log event template to represent a corresponding log event in the software log; and generating a vector representation of the log event template. Parsing the software log to generate a log event template to represent a corresponding log event in the software log may include identifying log events in the software log; and for each log event in the log event in the software log, extracting a constant portion (e.g., and discarding a variable portion), and converting the extracted constant portion of the log event in the software log to a given log event template.

[0121] In at least one implementation, the software log includes: an execution log generated by the execution of the software application; and / or a user log generated with the user executing the software application (e.g., in a customer environment). The user log may include at least some of the information characterizing a software problem associated with the software application.

[0122] In one or more implementations, a given test step vector representation is generated by aggregating one or more of the vector representations of the log event templates (e.g., as combined with Figure 8 Generating a test step vector representation of information representing a software problem may include parsing the information to generate a log event template to represent a corresponding log event in the information; and generating a vector representation of the log event template using a first mapping (e.g., in the form of a log event template dictionary). In addition, generating a test step vector representation of information representing a software problem may further include: identifying one or more additional test step vector representations (e.g., using a similarity measure of at least some of the test step vector representations) based on the similarity measure of at least some of the test step vector representations. Fig. 9 930), mapping one or more additional test step vector expressions associated with a given test step vector representation of information to a corresponding set of test step functions (e.g., using Fig. 9 940), and for a given test step vector representation in the information, a given test step function is selected from the corresponding test step function set (e.g., using Fig. 9 Test step function selection 950).

[0123] For example, combined with Figure 5 , Figure 8 , Fig. 9 and Fig.11 The specific processing operations and other network functions described are presented only by way of illustrative examples and should not be construed as limiting the scope of the present disclosure in any way. Alternative implementations may use other types of processing operations to provide functionality for automatically generating software application test cases for evaluating software application problems. For example, in other implementations, the ordering of process steps may vary, or certain steps may be performed simultaneously with each other rather than serially. In one aspect, the process may skip one or more of the actions. In other aspects, one or more of the actions are performed simultaneously. In some aspects, additional actions may be performed.

[0124] It should be understood that the specific advantages described above and elsewhere herein are associated with specific illustrative implementations and need not be present in other implementations. In addition, the specific types of information processing system features and functions shown in the drawings and described above are merely exemplary, and many other arrangements may be used in other implementations.

[0125] Now refer to Fig.12 and Fig.13, describes in more detail an illustrative implementation of a processing platform for implementing functionality for automatically generating software application test cases for evaluating software application problems. Although described in the context of information processing system 100, in other implementations, these platforms may also be used to implement at least portions of other information processing systems.

[0126] Fig.12 An example processing platform including cloud infrastructure 1200 is shown. Cloud infrastructure 1200 includes a plurality of processing platforms that can be used to implement Figure 1 The cloud infrastructure 1200 includes a combination of physical and virtual processing resources of at least a portion of the information processing system 100 in the cloud. The cloud infrastructure 1200 includes a plurality of virtual machines (VMs) and / or container groups 1202-1, 1202-2, ... 1202-L implemented using a virtualization infrastructure 1204. The virtualization infrastructure 1204 runs on a physical infrastructure 1205 and illustratively includes one or more hypervisors and / or operating system-level virtualization infrastructure. The operating system-level virtualization infrastructure illustratively includes a kernel control group of a Linux operating system or other type of operating system.

[0127] The cloud infrastructure 1200 further includes a plurality of groups of applications 1210-1, 1210-2, ... 1210-L running on respective ones of the VM / container groups 1202-1, 1202-2, ... 1202-L under the control of the virtualization infrastructure 1204. The VM / container group 1202 may include a respective VM, a respective group of one or more containers, or a respective set of one or more containers running in a VM.

[0128] exist Fig.12 In some implementations, the VM / container group 1202 includes corresponding VMs implemented using a virtualization infrastructure 1204 including at least one hypervisor. A hypervisor platform can be used to implement a hypervisor within the virtualization infrastructure 1204, wherein the hypervisor platform has an associated virtual infrastructure management system. The underlying physical machines can include one or more distributed processing platforms, which include one or more storage systems.

[0129] exist Fig.12 In other implementations, VM / container group 1202 includes corresponding containers implemented using virtualization infrastructure 1204 that provides operating system-level virtualization capabilities, such as support for Docker containers running on bare metal hosts or Docker containers running on VMs. The containers are illustratively implemented using corresponding kernel control groups of the operating system.

[0130] As described above, one or more of the processing modules or other components of information handling system 100 may each run on a computer, server, storage device, or other processing platform element. A given such element may be considered an example of what is more generally referred to herein as a "processing device." Fig.12 The cloud infrastructure 1200 shown may represent at least a portion of a processing platform. Another example of such a processing platform is Fig.13 Processing platform 1300 is shown.

[0131] The processing platform 1300 in this implementation comprises a portion of the information handling system 100 and includes a plurality of processing devices, denoted as 1302 - 1 , 1302 - 2 , 1302 - 3 , . . . 1302 -K, which communicate with each other via a network 1304 .

[0132] Network 1304 may include any type of network including, for example, a global computer network such as the Internet, a WAN, a LAN, a satellite network, a telephone or wired network, a cellular network, a wireless network such as a WiFi or WiMAX network, or various portions or combinations of these and other types of networks.

[0133] The processing device 1302 - 1 in the processing platform 1300 includes a processor 1310 coupled to a memory 1312 .

[0134] Processor 1310 may include a microprocessor, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), a video processing unit (VPU), or other types of processing circuits, as well as portions or combinations of such circuit elements.

[0135] The memory 1312 may include any combination of random access memory (RAM), read-only memory (ROM), flash memory, or other types of memory. The memory 1312 and other memories disclosed herein should be viewed as illustrative examples of what are more generally referred to as "processor-readable storage media" that store executable program code for one or more software programs.

[0136] An article of manufacture including such a processor-readable storage medium is considered an illustrative implementation. A given such article of manufacture may include, for example, a storage array, a storage disk, or an integrated circuit containing RAM, ROM, flash memory or other electronic memory, or any of a variety of other types of computer program products. The term "article of manufacture" as used herein should be understood to exclude transient propagating signals. Many other types of computer program products including a processor-readable storage medium may be used.

[0137] Also included in the processing device 1302-1 is network interface circuitry 1314 for interfacing the processing device with the network 1304 and other system components, and may include a conventional transceiver.

[0138] It is assumed that the other processing devices 1302 of the processing platform 1300 are configured in a manner similar to that shown in the figure for the processing device 1302-1.

[0139] Again, the specific processing platform 1300 shown in the figure is presented only by way of example, and the information processing system 130 may include additional or alternative processing platforms, as well as many different processing platforms in any combination. Each such platform includes one or more computers, servers, storage devices, or other processing devices.

[0140] For example, other processing platforms for implementing the illustrative implementations may include a converged infrastructure.

[0141] It should therefore be understood that in other implementations, different arrangements of additional or alternative elements may be used.At least a subset of these elements may be implemented together on a common processing platform, or each such element may be implemented on a separate processing platform.

[0142] As previously indicated, the components of the information processing system disclosed herein may be implemented at least in part in the form of one or more software programs stored in a memory and executed by a processor of a processing device. For example, at least a portion of the functionality disclosed herein for automating the generation of software application test cases for evaluating software application issues is illustratively implemented in the form of software running on one or more processing devices.

[0143] It should be emphasized again that the above implementation is presented for illustration purposes only. Many variations and other alternative implementations may be used. For example, the disclosed technology may be applied to various other types of information processing systems, software logs, test cases, etc. In addition, in other implementations, the specific configurations of the system and device elements illustratively shown in the drawings and the associated processing operations may vary. In addition, the various assumptions made in the process of describing the illustrative implementations above should also be regarded as exemplary, rather than requirements or limitations of the present disclosure. Many other alternative implementations within the scope of the appended claims will be clear to those skilled in the art.

Claims

1. A method comprising: obtaining a first mapping of a plurality of log event templates related to one or more log events in one or more software logs generated by executing a software application on one or more information technology assets of a plurality of information technology assets of an information technology infrastructure to respective ones of the vector representations of log event templates; obtaining a second mapping of a plurality of test step vector representations generated using the vector representation of the log event template to corresponding test step functions of a plurality of test step functions, wherein a given test step vector representation comprises one or more of the vector representations of the log event template, and wherein the second mapping is generated by analyzing execution of a plurality of test steps associated with the software application in an execution history of the one or more software logs; In response to obtaining information characterizing a software problem associated with the software application: generating one or more test step vector representations of the information characterizing the software problem using the first mapping; mapping the one or more test step vector representations of the information characterizing the software problem to corresponding test step functions of a plurality of test step functions using the second mapping; as well as Automatically generate a test case logic flow using the mapped test step functions to evaluate the software problem associated with the software application; Wherein the method is performed by at least one processing device comprising a processor coupled to a memory.

2. The method according to claim 1, wherein the first mapping is implemented using a log event template dictionary, the log event template dictionary being generated by the following steps: obtaining one or more software logs generated by executing the software application on one or more information technology assets of the plurality of information technology assets of the information technology infrastructure; parsing the one or more software logs to generate the plurality of log event templates to represent corresponding log events among the log events in the one or more software logs; as well as The vector representation of the plurality of log event templates is generated.

3. The method of claim 2, wherein parsing the one or more software logs to generate the plurality of log event templates to represent corresponding log events in the log events in the one or more software logs comprises: identifying a plurality of log events in the one or more software logs; And for each log event in the multiple log events in the one or more software logs, a constant part is extracted, and the extracted constant part of each log event in the multiple log events in the one or more software logs is converted into a given log event template in a plurality of log event templates.

4. The method of claim 1, wherein the one or more software logs include at least one of the following: one or more execution logs generated by said execution of said software application; and In conjunction with one or more user logs generated by one or more users executing the software application. 5 . The method of claim 4 , wherein the one or more user logs include at least some of the information characterizing the software problem associated with the software application. 6 . The method of claim 1 , wherein the given test step vector representation is generated by aggregating one or more of the vector representations of the log event template.

7. The method of claim 1 , wherein generating the one or more test step vector representations of the information characterizing the software problem comprises: parsing the information to generate a plurality of log event templates to represent corresponding log events in the log events in the information; And using the first mapping to generate vector representations of the plurality of log event templates.

8. The method of claim 1 , wherein generating the one or more test step vector representations of the information characterizing the software problem further comprises: One or more additional test step vector representations are identified based on a similarity measure for at least some of the one or more test step vector representations, the one or more additional test step vector representations associated with a given test step vector representation of the information are mapped to a corresponding set of test step functions, and a given test step function is selected from the corresponding set of test step functions for the given test step vector representation in the information.

9. An apparatus comprising: at least one processing device comprising a processor coupled to a memory; The at least one processing device is configured to implement the following steps: obtaining a first mapping of a plurality of log event templates related to one or more log events in one or more software logs generated by executing a software application on one or more information technology assets of a plurality of information technology assets of an information technology infrastructure to respective ones of the vector representations of log event templates; obtaining a second mapping of a plurality of test step vector representations generated using the vector representation of the log event template to corresponding test step functions of a plurality of test step functions, wherein a given test step vector representation comprises one or more of the vector representations of the log event template, and wherein the second mapping is generated by analyzing execution of a plurality of test steps associated with the software application in an execution history of the one or more software logs; In response to obtaining information characterizing a software problem associated with the software application: generating one or more test step vector representations of the information characterizing the software problem using the first mapping; mapping the one or more test step vector representations of the information characterizing the software problem to corresponding test step functions of a plurality of test step functions using the second mapping; as well as The mapped test step functions are used to automatically generate a test case logic flow to evaluate the software problem associated with the software application.

10. The apparatus of claim 9, wherein the first mapping is implemented using a log event template dictionary, the log event template dictionary being generated by: obtaining one or more software logs generated by executing the software application on one or more information technology assets of the plurality of information technology assets of the information technology infrastructure; parsing the one or more software logs to generate the plurality of log event templates to represent corresponding log events among the log events in the one or more software logs; as well as The vector representation of the plurality of log event templates is generated.

11. The apparatus of claim 10, wherein parsing the one or more software logs to generate the plurality of log event templates to represent corresponding log events in the log events in the one or more software logs comprises: identifying a plurality of log events in the one or more software logs; And for each log event in the multiple log events in the one or more software logs, a constant part is extracted, and the extracted constant part of each log event in the multiple log events in the one or more software logs is converted into a given log event template in a plurality of log event templates.

12. The apparatus of claim 9, wherein the one or more software logs include at least one of the following: one or more execution logs generated by said execution of said software application; and One or more user logs generated in conjunction with execution of the software application by one or more users, wherein the one or more user logs include at least some of the information characterizing the software problem associated with the software application.

13. An apparatus according to claim 9, wherein generating the one or more test step vector representations of the information characterizing the software problem includes parsing the information to generate multiple log event templates to represent corresponding log events in the log events in the information; and using a first mapping to generate vector representations of multiple log event templates.

14. The apparatus of claim 9, wherein generating the one or more test step vector representations of the information characterizing the software problem further comprises: One or more additional test step vector representations are identified based on a similarity measure for at least some of the one or more test step vector representations, the one or more additional test step vector representations associated with a given test step vector representation of the information are mapped to a corresponding test step function set, and a given test step function is selected from the corresponding test step function set for the given test step vector representation in the information.

15. A non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code, when executed by at least one processing device, causes the at least one processing device to perform the following steps: obtaining a first mapping of a plurality of log event templates related to one or more log events in one or more software logs generated by executing a software application on one or more information technology assets of a plurality of information technology assets of an information technology infrastructure to respective ones of the vector representations of log event templates; obtaining a second mapping of a plurality of test step vector representations generated using the vector representation of the log event template to corresponding test step functions of a plurality of test step functions, wherein a given test step vector representation comprises one or more of the vector representations of the log event template, and wherein the second mapping is generated by analyzing execution of a plurality of test steps associated with the software application in an execution history of the one or more software logs; In response to obtaining information characterizing a software problem associated with the software application: generating one or more test step vector representations of the information characterizing the software problem using the first mapping; mapping the one or more test step vector representations of the information characterizing the software problem to corresponding test step functions of a plurality of test step functions using the second mapping; as well as A test case logic flow is automatically generated using the mapped test step functions to evaluate the software problem associated with the software application.

16. The non-transitory processor-readable storage medium of claim 15, wherein the first mapping is implemented using a log event template dictionary, the log event template dictionary being generated by: obtaining one or more software logs generated by executing the software application on one or more information technology assets of the plurality of information technology assets of the information technology infrastructure; parsing the one or more software logs to generate the plurality of log event templates to represent corresponding log events among the log events in the one or more software logs; as well as The vector representation of the plurality of log event templates is generated.

17. The non-transitory processor-readable storage medium of claim 16, wherein parsing the one or more software logs to generate the plurality of log event templates to represent corresponding ones of the log events in the one or more software logs comprises: identifying a plurality of log events in the one or more software logs; And for each log event in the multiple log events in the one or more software logs, a constant part is extracted, and the extracted constant part of each log event in the multiple log events in the one or more software logs is converted into a given log event template in a plurality of log event templates.

18. The non-transitory processor-readable storage medium of claim 15, wherein the one or more software logs include at least one of: one or more execution logs generated by said execution of said software application; and One or more user logs generated in conjunction with execution of the software application by one or more users, wherein the one or more user logs include at least some of the information characterizing the software problem associated with the software application.

19. The non-transitory processor-readable storage medium of claim 15, wherein generating the one or more test step vector representations of the information characterizing the software issue comprises: parsing the information to generate a plurality of log event templates to represent corresponding log events in the log events in the information; And using the first mapping to generate vector representations of the plurality of log event templates.

20. The non-transitory processor-readable storage medium of claim 15, wherein said generating said one or more test step vector representations of said information characterizing said software problem further comprises: One or more additional test step vector representations are identified based on a similarity measure for at least some of the one or more test step vector representations, the one or more additional test step vector representations associated with a given test step vector representation of the information are mapped to a corresponding set of test step functions, and a given test step function is selected from the corresponding set of test step functions for the given test step vector representation in the information.

Images