Components for determining test step conflicts in firmware testing

Abstract

An information handling system stores a list of commands associated with test cases. The system receives the list of commands associated with the test case and provides the list of commands to an input layer of a state attribute probabilistic machine learning model. And an output layer of the state attribute probability machine learning model outputs a state attribute probability. The state attribute probability identifies an overlap of state attributes of test steps in the test case. Based on the state attribute probability, the system determines whether the test steps of the test case conflict. And in response to the fact that the test steps do not conflict, the system merges the test cases. In response to the test step conflict, the system stops merging the test cases.

Description

Technical Field

[0001] The present disclosure relates generally to information handling systems and, more particularly, to a component for determining test step conflicts in firmware testing. Background Art

[0002] As the value and use of information continue to grow, individuals and businesses seek additional ways to process and store information. One option is an information handling system. An information handling system typically processes, compiles, stores or transmits information or data for commercial, personal or other purposes. Technology and information handling needs and requirements may vary between different applications. Therefore, information handling systems may also differ in terms of what information is handled, how the information is handled, how much information is processed, stored or transmitted, and how quickly and efficiently the information can be processed, stored or transmitted. The changes in information handling systems allow information handling systems to be general or configured for specific users or specific purposes (such as financial transaction processing, airline ticketing, enterprise data storage or global communications). In addition, an information handling system may include a variety of hardware and software resources that can be configured to process, store and transmit information, and may include one or more computer systems, graphical interface systems, data storage systems, networking systems and mobile communication systems. Information handling systems can also implement various virtualization architectures. Data and voice communications between information handling systems can be carried out via a network, which is wired, wireless or a combination thereof. Summary of the invention

[0003] An information handling system includes a memory and a processor. The memory may store a command list for the information handling system, and the command list is associated with a test case. The processor may receive a command list associated with the test case. The processor may provide the command list to an input layer of a state attribute probability machine learning model. The output layer of the state attribute probability machine learning model may output state attribute probabilities. The state attribute probabilities identify overlaps of state attributes of test steps in the test case. Based on the state attribute probabilities, the processor may determine whether the test steps of the test case conflict. In response to the test steps not conflicting, the processor may merge the test cases. In response to the test steps conflicting, the processor may stop merging the test cases. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] It should be understood that for simplicity and clarity of illustration, the elements shown in the figures are not necessarily drawn to scale. For example, the dimensions of some elements may be exaggerated relative to other elements. Embodiments incorporating the teachings of the present disclosure are shown and described with respect to the drawings herein, in which:

[0005] Figure 1 is a block diagram of a general information handling system according to an embodiment of the present disclosure;

[0006] Figure 2 is a flowchart of a method for training a machine learning system to determine whether test steps of a test case conflict according to at least one embodiment of the present disclosure;

[0007] Figure 3 is a block diagram of a machine learning system according to at least one embodiment of the present disclosure;

[0008] Figure 4 is a flowchart of a method for determining whether test steps of a test case conflict according to at least one embodiment of the present disclosure; and

[0009] Figure 5 is a block diagram of a machine learning system according to at least one embodiment of the present disclosure.

[0010] The use of the same reference numbers in different drawings indicates similar or identical items. DETAILED DESCRIPTION

[0011] The following description in conjunction with the accompanying drawings is provided to help understand the teaching disclosed herein. The description focuses on the specific implementation and implementation of the teaching and is provided to help describe the teaching. This focus should not be interpreted as limiting the scope or applicability of the teaching.

[0012] Figure 1A generalized embodiment of an information handling system 100 according to an embodiment of the present disclosure is shown. For the purposes of the present disclosure, an information handling system may include any tool or tool set operable to calculate, classify, process, transmit, receive, retrieve, generate, switch, store, display, indicate, detect, record, reproduce, handle or utilize any form of information, intelligence or data for commercial, scientific, control, entertainment or other purposes. For example, the information handling system 100 may be a personal computer, a laptop computer, a smart phone, a tablet device or other consumer electronic device, a network server, a network storage device, a switch, a router or another network communication device, or any other suitable device, and may vary in size, shape, performance, functionality and price. The information handling system may include random access memory (RAM), one or more processing resources (such as a central processing unit (CPU) or hardware or software control logic), ROM and / or other types of non-volatile memory. Additional components of the information handling system may include one or more disk drives, one or more network ports for communicating with external devices, and various input and output (I / O) devices (such as a keyboard, mouse, touch screen and / or video display). In addition, the information handling system 100 may include processing resources for executing machine executable code, such as a central processing unit (CPU), a programmable logic array (PLA), an embedded device such as a system on a chip (SoC), or other control logic hardware. The information handling system 100 may also include one or more computer-readable media for storing machine executable code such as software or data. Additional components of the information handling system 100 may include one or more storage devices that can store machine executable code, one or more communication ports for communicating with external devices, and various input and output (I / O) devices (such as a keyboard, mouse, and video display). The information handling system 100 may also include one or more buses that are operable to transfer information between various hardware components.

[0013] Information handling system 100 may include devices or modules embodying one or more of the following devices or modules, and operate to perform one or more of the following methods. Information handling system 100 includes processors 102 and 104, input / output (I / O) interface 110, memory 120 and 125, graphics interface 130, basic input and output system / universal extensible firmware interface (BIOS / UEFI) module 140, disk controller 150, hard disk drive (HDD) 154, optical disk drive (ODD) 156, disk emulator 160 connected to external solid state drive (SSD) 162, I / O bridge 170, one or more extended resources 174, trusted platform module (TPM) 176, network interface 180, management device 190, and power supply 195. Processors 102 and 104, I / O interface 110, memory 120, graphics interface 130, BIOS / UEFI module 140, disk controller 150, HDD 154, ODD 156, disk emulator 160, SSD 162, I / O bridge 170, extended resources 174, TPM 176, and network interface 180 operate together to provide a host environment for information handling system 100, which operates to provide the data processing functionality of the information handling system. The host environment operates to execute machine executable code, including platform BIOS / UEFI code, device firmware, operating system code, applications, programs, etc., to perform data processing tasks associated with information handling system 100.

[0014] In a host environment, processor 102 is connected to I / O interface 110 via processor interface 106, and processor 104 is connected to I / O interface 110 via processor interface 108. Memory 120 is connected to processor 102 via memory interface 122. Memory 125 is connected to processor 104 via memory interface 127. Graphics interface 130 is connected to I / O interface 110 via graphics interface 132 and provides video display output 136 to video display 134. In a particular embodiment, information handling system 100 includes separate memories dedicated to each of processors 102 and 104 via separate memory interfaces. Examples of memories 120 and 130 include random access memory (RAM) such as static RAM (SRAM), dynamic RAM (DRAM), non-volatile RAM (NV-RAM), etc., read-only memory (ROM), another type of memory, or a combination thereof.

[0015] The BIOS / UEFI module 140, the disk controller 150, and the I / O bridge 170 are connected to the I / O interface 110 via an I / O channel 112. Examples of the I / O channel 112 include a peripheral component interconnect (PCI) interface, a PCI-Extension (PCI-X) interface, a high-speed PCI-Express (PCIe) interface, another industry standard or proprietary communication interface, or a combination thereof. The I / O interface 110 may also include one or more other I / O interfaces, including an Industry Standard Architecture (ISA) interface, a Small Computer Serial Interface (SCSI) interface, an Inter-Integrated Circuit (IIC) interface, and a SCSI interface. 2 C) interface, system packet interface (SPI), universal serial bus (USB), another interface, or a combination thereof. BIOS / UEFI module 140 includes BIOS / UEFI code that is operable to detect resources within information handling system 100, provide drivers for resources, initialize resources, and access resources. BIOS / UEFI module 140 includes code that is operable to detect resources within information handling system 100, provide drivers for resources, initialize resources, and access resources.

[0016] The disk controller 150 includes a disk interface 152 that connects the disk controller to the HDD 154, the ODD 156, and the disk emulator 160. Examples of the disk interface 152 include an integrated drive electronics (IDE) interface, an advanced technology attachment (ATA) such as a parallel ATA (PATA) interface or a serial ATA (SATA) interface, a SCSI interface, a USB interface, a proprietary interface, or a combination thereof. The disk emulator 160 allows the SSD 164 to be connected to the information handling system 100 via the external interface 162. Examples of the external interface 162 include a USB interface, an IEEE 4394 (Firewire) interface, a proprietary interface, or a combination thereof. Alternatively, the solid state drive 164 can be disposed within the information handling system 100.

[0017] I / O bridge 170 includes a peripheral interface 172 that connects the I / O bridge to an extended resource 174, a TPM 176, and a network interface 180. Peripheral interface 172 may be an interface of the same type as I / O channel 112, or may be an interface of a different type. Thus, when peripheral interface 172 and I / O channel 112 are of the same type, I / O bridge 170 extends the capabilities of the I / O channel, and when they are of different types, the I / O bridge converts information from a format suitable for the I / O channel to a format suitable for peripheral bridge 172. Extended resource 174 may include a data storage system, an additional graphics interface, a network interface card (NIC), a sound / video processing card, another extended resource, or a combination thereof. Extended resource 174 may be located on a main circuit board, on a separate circuit board or expansion card disposed within information handling system 100, on a device external to the information handling system, or a combination thereof.

[0018] The network interface 180 represents a NIC that is disposed within the information handling system 100, located on the main circuit board of the information handling system, integrated into another component such as the I / O interface 110, located in another suitable location, or a combination thereof. The network interface device 180 includes network channels 182 and 184 that provide interfaces to devices outside the information handling system 100. In a specific embodiment, the network channels 182 and 184 are of a different type from the peripheral channels 172, and the network interface 180 converts information from a format suitable for the peripheral channels to a format suitable for the external device. Examples of network channels 182 and 184 include InfiniBand channels, fiber channel channels, Gigabit Ethernet channels, proprietary channel architectures, or a combination thereof. Network channels 182 and 184 can be connected to external network resources (not shown). Network resources can include another information handling system, a data storage system, another network, a grid management system, another suitable resource, or a combination thereof.

[0019] Management device 190 represents one or more processing devices that operate together to provide a management environment for information handling system 100, such as a dedicated baseboard management controller (BMC) system-on-chip (SoC) device, one or more associated memory devices, one or more network interface devices, complex programmable logic devices (CPLD), etc. Specifically, management device 190 is connected to various components of the host environment via various internal communication interfaces (such as low pin count (LPC) interface, inter-integrated circuit (I2C) interface, PCIe interface, etc.) to provide an out-of-band (OOB) mechanism to retrieve information related to the operation of the host environment, provide BIOS / UEFI or system firmware updates, and manage non-processing components of information handling system 100, such as system cooling fans and power supplies. Management device 190 may include a network connection to an external management system, and the management device may communicate with the management system to report status information of information handling system 100, receive BIOS / UEFI or system firmware updates, or perform other tasks for managing and controlling the operation of information handling system 100.

[0020] The management device 190 may operate on a separate power plane from components of the host environment, such that the management device receives power to manage the information handling system 100 when the information handling system is otherwise shut down. Examples of the management device 190 include commercially available BMC products or other devices that operate in accordance with the Intelligent Platform Management Initiative (IPMI) specification, the Web Services Management (WSMan) interface, the Redfish application programming interface (API), another Distributed Management Task Force (DMTF), or other management standards, and may include an integrated Dell Remote Access Controller (iDRAC), an embedded controller (EC), etc. The management device 190 may also include associated memory devices, logic devices, security devices, etc., as needed or desired.

[0021] Figure 2 The flow of a method 200 for training a machine learning system to determine whether test steps of a test case conflict according to at least one embodiment of the present disclosure is shown, and the method begins at block 202. In an example, the method 200 may include but is not limited to Figure 1 It should be readily understood that not every method step set forth in the flowchart is always necessary, but rather that certain steps of the method may be combined, performed simultaneously, in a different order, or may be omitted without departing from the scope of the present disclosure.

[0022] At box 204, a test case log is collected. In an example, the test case log can be associated with different test cases of the firmware test data. In some examples, the firmware test can include time-intensive steps based on interactions with hardware components within the information handling system. One such time-intensive test step can be a system restart of the information handling system. The amount of time consumed by the firmware test can be reduced by merging different test steps together. However, if conflicting test steps are merged, the firmware test may become unreliable. In some examples, the test steps of the firmware test may conflict based on a variety of reasons. For example, a command in a test step may change one or more hidden states of the firmware, which in turn may affect the results of subsequent commands.

[0023] At block 206, the execution commands for the firmware test are extracted from the test case log. At block 208, the extracted commands are decoded into readable text. In some examples, the execution commands can be any suitable command format, such as binary / hexadecimal commands. After the commands have been decoded, the test step (SET) commands and test step groups (GET commands) are identified at block 210. At block 212, all SET and GET commands are obtained.

[0024] At block 214, the SET and GET commands are vectorized. In some examples, each individual SET(S) command and individual GET(G) command may be vectorized into a different vector. In an example, the vectorization of the SET(S) command and the GET(G) command may create a matrix for each individual SET(S) command and individual GET(G) command. At block 216, the probability of each GET and SET vector is calculated. In an example, the probability may be a discounted observation motivation probability p i For each GET vector G i To SET vector S j Compute discounted observed motivation probabilities where i>j.

[0025] At block 218, the GET vectors are grouped into a plurality of state attribute groups or clusters. In an example, the GET vectors G may be clustered into any suitable number k of state attribute groups. In some examples, each state attribute group may be a different state attribute for a firmware update, including but not limited to an idle state, a download state, a downloaded state, a success state, an ongoing state, and an insufficient memory state, a connection lost state, a verification failed state, an unsupported image state, and an invalid location state. In an example, the training data may include z GET vectors. In some examples, the z vectors may be clustered into w attribute categories. Thus, each G i can all be marked as scalar category identifiers.

[0026] At block 220, each GET vector is mapped to a one-hot state attribute vector a i In an example, the one-hot state attribute vector can be of any suitable size, such as (1×k), and k can be the number of different attribute state groups or clusters. i Mapped to the one-hot state attribute vector a i , the mapping can be defined as GET command → G i →a i At block 222, the state attribute probability of each SET vector is calculated. In an example, if the test case has n SET and GET commands, the state attribute probability affected by the jth SET command can be calculated using the following equation 1.

[0027]

[0028] In equation 1, A j is the state attribute probability, p i is the discounted observation motivation probability of the i-th GET command. In the example, if the i-th command is SET, then p i = 0. Similarly in equation 1, a i is the one-hot state vector of the state attributes of the i-th GET command. In the example, a i The size of is 1X w, and w is the count of total state attribute labels.

[0029] At box 224, a training data set is generated. In an example, the training data set can be any suitable data generated from commands of a test case log, such as a SET (S) command and a corresponding state attribute probability (A). In some examples, the training data set can be a SET command-state attribute probability pair (S-A pair). The S-A pair can be extracted and generated from the test case execution log via a discounted observation motivation probability operation. At box 226, a neural network model is trained based on the training data set, and the process ends at box 228. In an example, the neural network model can be trained to use the state attribute probability to determine whether the test steps conflict.

[0030] Figure 3 300 according to at least one embodiment of the present disclosure. In an example, the machine learning system can be any suitable machine learning model, such as a neural network model. The machine learning system 300 can be substantially similar to the Figure 2 The machine learning system 300 includes an input layer 302, one or more hidden layers 304, and an output layer 306. The input layer 302 can receive information from an information processing system (such as Figure 1The hidden layer 304 may be configured to receive any suitable data associated with the information handling system 100 of the machine learning system 300 and provide the data to the hidden layer 304. In an example, the test case log may be used as input data for the input layer 302 of the machine learning system 300. The hidden layer 304 may perform one or more operations on the input data, such as SET commands and GET commands in the test case log, to calculate the state attribute probabilities of the test steps. In some examples, the hidden layer 304 may determine whether the test steps of the firmware update conflict.

[0031] During the training of the machine learning system 300, the test case log can be used to calculate the probability of conflicts between the test steps of the firmware update. In some examples, the training of the hidden layer 304 can establish a relationship between the test steps (SET commands) that will result in changing the state of the information handling system and the state attributes of the information handling system. By using the proposed "discounted observation motivation probability" on a set of test steps (GET commands) where the relevant state attributes may be observed, the state attributes can be hidden externally. In the example, the training of the hidden layer 304 can be performed in any suitable manner, including but not limited to supervised learning, unsupervised learning, reinforcement learning, and self-learning. In the example, any machine learning model including but not limited to a linear regression model can be utilized to determine the user experience.

[0032] Figure 4 The flow of a method 400 for determining whether test steps of a test case conflict according to at least one embodiment of the present disclosure is shown, and the method begins at block 402. In an example, the method 400 may include but is not limited to Figure 1 It should be readily understood that not every method step set forth in the flowchart is always necessary, but rather that certain steps of the method may be combined, performed simultaneously, in a different order, or may be omitted without departing from the scope of the present disclosure.

[0033] At block 404, a SET command list is received from a test case. In an example, the SET command list in the test case may be test steps to be performed during a firmware update. At block 406, the commands are decoded into readable text. In an example, an Intelligent Platform Management Interface (IPMI) may be a module for communicating with an information handling system such as a Figure 1The IPMI command may be in hexadecimal format, such as a GET command in hexadecimal format may be 0x30 0xd1 0x0 0x9 0x1 0x00. In this example, the text data of the GET command may be OEM GET_FRU_LED_STATE 0 9 1FRU_MARKER. Similarly, the hexadecimal SET command may be 0x30 0xd00x0 0x9 0x1 0x00 0x00 0x00 0x00. In this example, the text data of the SET command may be OEM SET_FRU_LED_STATE 0 91FRU_MARKER ON_DEFAULT OFF_DEFAULT FIRMWARE.

[0034] At block 408, the commands are vectorized into a matrix (A). In some examples, SET and GET commands may be vectorized in any suitable manner. For example, commands such as IPMI commands may be preprocessed by skipping specific words (such as numbers), lowercase letters, splitting strings, and vectorization methods known in the art. In an example, the i-th GET command may be converted into a vector G of size 1×v i , and v can be the number of all feature words in the GET command. The jth SET command can be converted into a vector S of size 1X u j , and u can be the number of all feature words in the SET command.

[0035] At block 410, the matrix (A) is input into the machine learning model and a second matrix (A) is output. l In an example, the machine learning model can be substantially similar to Figure 2 At box 412, the second matrix (A l ) is binarized. In an example, the threshold may be a discriminator threshold t. In an example, the discriminator threshold t may be a hyperparameter, and the matrix A may be binarized as a matrix (A) via the following equation 2 l or A').

[0036] A' ij =1If A ij >t, otherwise A' ij =0Equation 2

[0037] At block 414, the second matrix (A l ) are summed element-wise over all rows to create a new vector (A l ”). At block 416, the new vector (A) is thresholded by 0. l”) is binarized. At block 418, the total number of test cases is summed. In the example, the sum of n test cases can be calculated using the following equation 3.

[0038]

[0039] At box 420, it is determined whether the sum A'' is greater than or equal to 1. If the sum is not greater than or equal to 1, then at box 422, it is determined that there are no conflicts between the test steps of the test case. At box 424, the test cases are merged, and the process ends at box 426. If it is determined at box 420 that the sum is greater than or equal to 1, then at box 428, it is determined that there are conflicts between the test steps of the test case. At box 430, the test cases are not merged, and the process ends at box 426.

[0040] Figure 5 5 shows a machine learning system 500 according to at least one embodiment of the present disclosure. The machine learning system 500 includes an input layer 502, one or more hidden layers 504, and an output layer 506. The machine learning system 500 may be substantially similar to Figure 2 The machine learning model 200 of FIG. 4 can receive any suitable data associated with the information handling system, such as test steps from test case 510 and test steps from test case 512. In an example, test case 510 can include any suitable number of SET commands 510, and test case 512 can include any suitable number of SET commands 522. In an example, method 400 can include, but is not limited to, Figure 1 may be executed by any suitable component of processor 102 or 104. Figure 5 Only two test cases 510 and 512 are shown, but the machine learning system 500 can perform these operations on any suitable number of test cases without departing from the scope of the present disclosure.

[0041] During execution of the machine learning system 500, the input layer 502 may receive the SET commands 520 from the test case 510 and the SET commands from the test case 512 and provide the SET commands to the hidden layer 504 in any suitable manner. For example, the input layer 502 may convert the SET commands 520 and 522 into human-readable text, vectorize each of the SET commands separately, and provide different matrices to the hidden layer 504. The hidden layer 504 may then cluster the GET commands from each of the test cases 510 and 512 into a plurality of state attribute groups. For example, the GET commands in the SET commands 520 may be clustered into any suitable number of groups, including, but not limited to, the state attribute groups 530, 532, and 534. Similarly, the GET commands in the SET commands 522 may be clustered into any suitable number of groups, including, but not limited to, the state attribute groups 540, 542, and 544.

[0042] In an example, hidden layer 504 can calculate state attribute probabilities 550 for state attribute groups 530, 532, and 534, and calculate state attribute probabilities 560 for state attribute groups 540, 542, and 544. State attribute probabilities 550 can include different probabilities for any suitable number of state attributes (such as state attribute 1, state attribute 2, state attribute 3, state attribute 4, and state attribute k). State attribute probabilities 560 can include different probabilities for any suitable number of state attributes (such as state attribute 1, state attribute 2, state attribute 3, state attribute 4, and state attribute k). In some examples, a processor (such as Figure 1 The processor 102 or 104 of the information handling system 100 in FIG. 1 may estimate the state attribute probability matrix (A) 550 and 560 based on the list of SET command vectors S. In an example, the estimated state attribute probability matrix (A) may be calculated by the following equation 4.

[0043] A=f() Equation 4

[0044] In equation 4 above, A is an estimated state attribute probability matrix, S is a SET command vector, and function f is learned by a neural network (such as neural network 500). For example, function f can determine the estimated state attribute probability using discounted observation motivation probability. In an example, a test case may contain many SET and GET commands. However, there may not be any explicit information to describe which GET command can be executed in a specific order to observe the changing state attributes that may be affected by a specific SET command. The metric (such as how a GET command may be affected by a specific SET command) can be defined as the observation motivation probability. In an example, a discount model can be used to calculate the observation motivation probability from a sequence of SET commands and GET commands. In some examples, the observation motivation probability can decrease exponentially based on the distance between a specific SET command and the corresponding GET command in the test case.

[0045] In some examples, the observed motivation distance between the SET command and the GET command can be defined or calculated in any suitable manner. For example, the observed motivation distance can be determined by merging adjacent SET commands into a single SET command and determining the motivation distance from the GET command to the SET command based on the number of SET commands between the GET command and the SET command. An exemplary command sequence in a test case (such as test case 510) is shown in Table 1 below.

[0046] Table 1: Command sequence

[0047]

[0048]

[0049] In Table 1, a processor (such as Figure 1 The processor 102 or 104 of the information handling system 100 in the command sequence may merge SET commands 2 and 3, and the resulting SET commands become SET-1', SET-2', and SET-3'. Table 1 shows that the distance between a GET command and a previous SET command in a command sequence is based on the number of SET commands between the GET command and the previous SET command. For example, the distance (d) between the GET command in Cmd idx 2 and SET-1' is 0, the distance (d) between the GET command in Cmd idx5 and SET-1' is 1, the distance (d) between the GET command in Cmd idx 6 and SET-1' is 1, the distance (d) between the GET command in Cmd idx 7 and SET-1' is 1, the distance (d) between the GET command in Cmd idx 9 and SET-1' is 2, and the distance (d) between the GET command in Cmd idx 10 and SET-1' is 2.

[0050] The distance (d) between the GET command in cmd idx 5 and SET-2' is 0, the distance (d) between the GET command in cmd idx 6 and SET-2' is 0, the distance (d) between the GET command in cmd idx 7 and SET-2' is 0, the distance (d) between the GET command in cmd idx 9 and SET-2' is 1, and the distance (d) between the GET command in cmd idx 10 and SET-2' is 1. The distance (d) between the GET command in cmd idx 5 and SET-3 is 0, the distance (d) between the GET command in cmd idx 6 and SET-3 is 0, the distance (d) between the GET command in cmd idx 7 and SET-3 is 0, the distance (d) between the GET command in cmd idx 9 and SET-3 is 1, and the distance (d) between the GET command in cmd idx 10 and SET-3 is 1. The distance (d) of the GET command in cmd idx 9 from SET-4 is 0, and the distance (d) of the GET command in cmd idx 10 from SET-4 is 0.

[0051] In some examples, the discounted observation motivation probability of each GET command can be defined or calculated in any suitable manner. For example, the discounted observation motivation probability can be determined by the following equation 5.

[0052] p=αXγ d Equation 5

[0053] In the above equation 5, α is the base probability, which is a hyperparameter between (0, 1). In addition, γ is the discount rate, which is a hyperparameter between (0, 1). An exemplary command sequence in a test case (such as test case 510, where α=1 and γ=0.5) is shown in Table 2 below.

[0054] Table 2: Command sequence

[0055]

[0056] In an example, the command sequence in Table 2 (such as a sequence of SET commands and GET commands) may be substantially similar to the command sequence in Table 1. In some examples, a processor (such as Figure 1The processor 102 or 104 of the information handling system 100 in Table 2 may calculate the discounted observation motivation probability for each GET command in Table 2. For example, the discounted observation motivation probability (p) associated with SET-1' for the GET command in Cmd idx 2 is 1, the discounted observation motivation probability (p) associated with SET-1' for the GET command in Cmd idx 5 is 0.5, the discounted observation motivation probability (p) associated with SET-1' for the GET command in Cmd idx 6 is 0.5, the discounted observation motivation probability (p) associated with SET-1' for the GET command in Cmd idx 7 is 0.5, the discounted observation motivation probability (p) associated with SET-1' for the GET command in Cmd idx 9 is 0.25, and the discounted observation motivation probability (p) associated with SET-1' for the GET command in Cmd idx 10 is 0.25.

[0057] The GET command in cmd idx 5 has a discounted observation motivation probability (p) associated with SET-2' of 1, the GET command in cmd idx 6 has a discounted observation motivation probability (p) associated with SET-2' of 1, the GET command in cmd idx 7 has a discounted observation motivation probability (p) associated with SET-2' of 1, the GET command in cmd idx 9 has a discounted observation motivation probability (p) associated with SET-2' of 0.5, and the GET command in cmd idx 10 has a discounted observation motivation probability (p) associated with SET-2' of 0.5. The GET command in cmd idx 5 has a discount observation motivation probability (p) associated with SET-3 of 1, the GET command in cmd idx 6 has a discount observation motivation probability (p) associated with SET-3 of 1, the GET command in cmd idx 7 has a discount observation motivation probability (p) associated with SET-3 of 1, the GET command in cmd idx 9 has a discount observation motivation probability (p) associated with SET-3 of 0.5, and the GET command in cmd idx 10 has a discount observation motivation probability (p) associated with SET-3 of 0.5. The GET command in cmd idx 9 has a discount observation motivation probability (p) associated with SET-4 of 1, and the GET command in cmd idx 10 has a discount observation motivation probability (p) associated with SET-4 of 1.

[0058] As described above, the hidden layer 504 can use the discounted observation motivation probability (p) of the GET command to calculate the state attribute probabilities 550 and 560. In an example, the hidden layer 504 can use the state attribute probabilities 550 and 560 to determine whether there is any test step overlap 570 between the test step 520 of the test case 510 and the test step 522 of the test case 512. Based on whether there is an overlap 570 between the test step 520 of the test case 510 and the test step 522 of the test case 512, the hidden layer 504 can provide an output layer 506 whether the SET command conflicts. In an example, given n different test cases S from l (1=[1…n]) SET command list, hidden layer 504 can generate A' l (l=[1…n]) list, where A' l is a matrix. Based on the determined A', the hidden layer 504 can be l Sum all rows in A' and binarize the matrix by thresholding it to 0 l Converted into another matrix A" l To obtain a separate affected state attribute vector for each test case. In the example, if the total affected state attribute vector is: Based on this calculation, if (A'')>1, the SET commands conflict. However, if (A'')≤1, the SET commands do not conflict.

[0059] If there is an overlap 570, a conflict notification 580 may be provided at the output layer 506. In response to the conflict notification 580, a processor (such as Figure 1 The processor 102 or 104 of the information handling system 100 may prevent or terminate the merging of the test steps. If there is no overlap 570, a non-conflict notification 582 may be provided at the output layer 506. In response to the non-conflict notification 582, the processor (such as Figure 1 The processor 102 or 104 of the information handling system 100 may incorporate the testing steps.

[0060] Although only a few exemplary embodiments are described in detail herein, those skilled in the art will readily appreciate that many modifications may be made to the exemplary embodiments without materially departing from the novel teachings and advantages of the embodiments of the present disclosure. Therefore, all such modifications are intended to be included within the scope of the embodiments of the present disclosure as defined by the appended claims. In the claims, means-plus-function clauses are intended to cover structures described herein as performing the recited function, and not only structural equivalents, but also equivalent structures.

Claims

1. An information processing system, comprising: a memory configured to store a command list for the information handling system, wherein the command list is associated with a plurality of test cases; as well as A processor, the processor communicating with the memory, the processor: receiving the command list associated with the test case; providing data associated with the command list to an input layer of a state attribute probabilistic machine learning model; Outputting state attribute probabilities by an output layer of the state attribute probability machine learning model, wherein the state attribute probabilities identify overlaps of state attributes of test steps in the test case; Based on the state attribute probability, determining whether the test steps of the test case conflict; In response to the test steps not conflicting, merging the test cases; as well as In response to the test step conflict, merging the test case is stopped.

2. The information handling system of claim 1 , wherein before receiving the command list, the processor further: Retrieving a test case log, wherein the test case log includes a plurality of execution commands; Calculating state attribute probabilities based on the execution commands from the test case log; Generate training dataset; as well as The state attribute probability machine learning model is trained based on the training data set.

3. The information handling system of claim 2, wherein before providing the data associated with the command list to the state attribute probabilistic machine learning model, the processor further: extracting a command from the command list; decoding the command into readable text; and The test step commands in the commands are vectorized into a matrix, wherein the data associated with the command list provided to the state attribute probabilistic machine learning model includes the matrix.

4. The information handling system of claim 1 , wherein determining whether the test steps of the test case conflict comprises the processor further: receiving an output matrix from the state-attribute probability machine learning model; and binarizing the output matrix.

5. The information handling system of claim 4, wherein the processor further: summing all rows of the output matrix element-wise to create a new vector; Binarizing the new vector; and The total number of test cases in the new vector is summed. 6 . The information handling system of claim 5 , wherein when the sum is less than 1, the testing steps do not conflict.

7. The information handling system of claim 5, wherein when the sum is greater than 1, the testing step conflicts.

8. The information handling system of claim 1, wherein the commands in the command list are test step commands.

9. A method comprising: receiving, by a processor of an information handling system, a command list associated with a plurality of test cases; providing data associated with the command list to an input layer of a state attribute probabilistic machine learning model; Outputting state attribute probabilities by an output layer of the state attribute probability machine learning model, wherein the state attribute probabilities identify overlaps of state attributes of test steps in the test case; Based on the state attribute probability, determining whether the test steps of the test case conflict; In response to the test steps not conflicting, merging the test cases; as well as In response to the test step conflict, merging the test case is stopped.

10. The method according to claim 9, wherein before receiving the command list, the method further comprises: Retrieving a test case log, wherein the test case log includes a plurality of execution commands; Calculating state attribute probabilities based on the execution commands from the test case log; Generate training dataset; as well as The state attribute probability machine learning model is trained based on the training data set.

11. The method of claim 10, wherein before providing the data associated with the command list to the state attribute probabilistic machine learning model, the method further comprises: extracting a command from the command list; decoding said command into readable text; as well as The test step commands in the commands are vectorized into a matrix, wherein the data associated with the command list provided to the state attribute probabilistic machine learning model includes the matrix.

12. The method according to claim 9, wherein determining whether the test steps of the test case conflict comprises: receiving an output matrix from the state-attribute probabilistic machine learning model; And binarizing the output matrix.

13. The method according to claim 12, further comprising: summing all rows of the output matrix element-wise to create a new vector; Binarizing the new vector; as well as The total number of test cases in the new vector is summed. The method of claim 13 , wherein when the sum is less than 1, the testing steps are non-confrontational. The method of claim 13 , wherein when the sum is greater than 1, the testing step conflicts. The method according to claim 9 , wherein the commands in the command list are test step commands.

17. A method comprising: receiving, by a processor of an information handling system, a command list associated with a plurality of test cases; extracting a command from the command list; decoding said command into readable text; vectorizing the test step commands in the commands into a matrix; providing the matrix associated with the command list to an input layer of a state attribute probabilistic machine learning model; Outputting state attribute probabilities by an output layer of the state attribute probability machine learning model, wherein the state attribute probabilities identify overlaps of state attributes of test steps in the test case; Determining whether the test steps of the test case conflict, wherein if the state attribute probability is greater than or equal to 1, the test steps conflict, and if the state attribute probability is less than 1, the test steps do not conflict; as well as If the test steps do not conflict, then merge the test cases; otherwise, stop merging the test cases.

18. The method according to claim 17, wherein before receiving the command list, the method further comprises: Retrieving a test case log, wherein the test case log includes a plurality of execution commands; Calculating state attribute probabilities based on the execution commands from the test case log; Generate training dataset; as well as The state attribute probability machine learning model is trained based on the training data set.

19. The method of claim 17, wherein determining whether the test steps of the test case conflict comprises: receiving an output matrix from the state-attribute probabilistic machine learning model; And binarizing the output matrix.

20. The method of claim 19, further comprising: summing all rows of the output matrix element-wise to create a new vector; Binarizing the new vector; as well as The total number of test cases in the new vector is summed.

Images