Method, computer system, and computer program for optimizing data collection for defect analysis

Abstract

To provide a method, a computer program product and a computer system for optimizing data collection for defect analysis.SOLUTION: The method includes determining a pre-defined defect category for the indicated defect and obtaining current dataset metadata for the pre-defined defect category. The dataset metadata includes a defined content for the dataset and a defined retrieval order for elements of the dataset, and is learned by monitoring dataset retrieval from end systems during a debugging process. The method also includes loading a dataset from the end system for use by a debugging tool. The datasets have predefined content and are loaded in a predefined order according to the dataset metadata.SELECTED DRAWING: Figure 2A

Description

[Technical Field]

[0001] The present invention relates to data collection for defect analysis, and more particularly to optimizing data collection using continuous learning for defect analysis.

[0002] A standard method for debugging computer system defects involves the use of debugging tools that are capable of analyzing data collected from the computer system. Traditional debugging methods involve collecting static data sets from the computer system and debugging the data using debugging tools. Typically, some degree of programming and / or configuration intervention is required to modify the static data sets collected for debugging purposes. Summary of the Invention [Problem to be solved by the invention]

[0003] For a given defect being debugged to find the root cause, there may be an optimal data set required for successful debugging. This data set can be defined in several ways. In one approach, all data is always collected for all defects being debugged. In another approach, a fixed data set is collected for a given category of defect. For example, if an assert occurs in a particular component of the system, only the data associated with that component is offloaded. [Means for solving the problem]

[0004] According to aspects of the present invention, computer-implemented methods, systems, and computer program products are provided as defined in the claims.

[0005] According to one embodiment of the present invention, a computer-implemented method for optimizing data collection for defect analysis is provided, comprising: determining a predefined defect category for a notified defect; acquiring current dataset metadata for the predefined defect category, where the dataset metadata includes predefined content for the dataset and a predefined acquisition order for elements of the dataset, where the dataset metadata is learned by monitoring dataset acquisition from an end system during a debug process; and providing a dataset from the end system for use by a debug tool, where the dataset has predefined content and is loaded in a predefined order according to the dataset metadata.

[0006] The described method has the advantage of providing an optimal data set and data set ordering over time to collect the data necessary to successfully and efficiently debug defects.

[0007] The method may include generating, using a learning process, metadata for each defined category, the metadata including a defined content for the dataset and a defined retrieval order for elements of the dataset.

[0008] Determining a predefined defect category for the notified defect may include providing a core dataset for defect categorization; and classifying the notified defect by comparing the notified defect to the core dataset. Comparing the notified defect to the core dataset may include comparing characteristics of the notified defect to the core dataset.

[0009] The method may include, during operation of a debug tool for the defect category, monitoring dataset accesses, including accesses to predefined content for current dataset metadata for the defect category and accesses to additional content required by the debug tool; maintaining a record of dataset accesses; and applying a learning process to update the dataset metadata for the defect category. The method may include applying a learning process to update the dataset metadata and deleting dataset content that has not been accessed for a predetermined number of defect category instances. The method may include monitoring a dataset access order for the predefined content and an order of access to any other additional content; maintaining a record of dataset access order; and applying a learning process to update the dataset metadata for the defect category.

[0010] The initial dataset metadata for the defect category can define the overall contents of the dataset and may be reduced over time to an optimal dataset. The defined acquisition order for elements of the dataset may include elements of discrete data stored in a data log or memory dump. The method can include generating additional operational metadata associated with the defect category, including a history of data access requests and a history of the order in which data access was requested.

[0011] According to another embodiment of the present invention, there is provided a system for optimizing data collection for defect analysis, comprising: a processor and a memory configured to provide the processor with computer program instructions for performing the functions of each component; a category determination component that determines a predefined defect category for a notified defect; a metadata acquisition component that acquires current dataset metadata for the predefined defect category, where the dataset metadata includes predefined content for the dataset and a predefined acquisition order for elements of the dataset, where the dataset metadata is learned by monitoring dataset acquisition from an end system during a debug process; and a dataset provision component that provides a dataset from the end system for use by a debug tool, where the dataset has predefined content and is loaded in a predefined order according to the dataset metadata.

[0012] According to one embodiment of the present invention, a computer program product for optimizing data collection for defect analysis is provided, the computer program product comprising a computer-readable storage medium having program instructions embodied thereon, the program instructions being executable by a processor to cause the processor to determine a predefined defect category for a notified defect; obtain current dataset metadata for the predefined defect category, the dataset metadata including predefined content for the dataset and a predefined acquisition order for elements of the dataset, the dataset metadata being learned by monitoring dataset acquisition from an end system during a debug process; and provide a dataset from the end system for use by a debug tool, the dataset having the predefined content and being loaded in the predefined order according to the dataset metadata.

[0013] The computer readable storage medium may be a non-transitory computer readable storage medium, and the computer readable program code may be executable by a processing circuit. [Brief explanation of the drawings]

[0014] Embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings, in which: [Figure 1] FIG. 1 is a block diagram of an exemplary embodiment of a system according to an embodiment of the present invention. [Figure 2A] FIG. 2 is a flow diagram of an exemplary embodiment of one aspect of a method according to an embodiment of the present invention. [Figure 2B] FIG. 10 is a flow diagram of an exemplary embodiment of another aspect of a method according to an embodiment of the present invention. [Figure 3] 3A and 3B are schematic diagrams illustrating simplified examples of data access requests over time used to learn datasets and dataset ordering for defect categories, according to aspects of the described method. [Figure 4] 4A and 4B are schematic diagrams illustrating data access requests over time used to learn data sets and data set ordering for defect categories, according to aspects of the described method. [Figure 5] FIG. 1 is a block diagram of an exemplary embodiment of a system according to an embodiment of the present invention. [Figure 6] FIG. 1 is a block diagram of an exemplary embodiment of a computing environment for executing at least a portion of the computer code associated with practicing the present invention.

[0015] It will be appreciated that for simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn to scale. For example, the dimensions of some of the elements may be exaggerated relative to other elements for clarity. Furthermore, where considered appropriate, reference numerals may be repeated among the figures to indicate corresponding or analogous features. DETAILED DESCRIPTION OF THE INVENTION

[0016] Method, system, and computer program product embodiments are provided for optimizing data collection for defect analysis using one or more debug tools.

[0017] Over time, the optimal set of data that needs to be collected for a given class of defect and the order in which that data set is accessed is learned. This provides a method for continuously monitoring human behavior and enhances the system's ability to provide the optimal data sets and data set ordering needed to debug a particular class of defect, allowing the debug system to collect that data and successfully and efficiently debug the defect.

[0018] Instead of always collecting a complete set of data logs for each defect instance, partial data logs are collected, thereby reducing the time required to obtain the required set of data logs. An optimal sequence for data log collection increases the likelihood of reaching the root cause of the defect in a shorter time. These criteria are dynamically modified iteratively to maintain optimal debug capability indefinitely.

[0019] Data collection for defect analysis is an improvement in the art of computer failure analysis generally, and more specifically, in the art of efficient access to data for use in defect analysis.

[0020] Referring to FIG. 1, a block diagram illustrates an exemplary embodiment of a system 100 that includes an intelligent data collection system 120 .

[0021] System 100 includes end-user system 140, which is the target system that contains memory dumps and other types of data collected for debugging. System 100 also includes a tool suite, debug tools 150, which provide access to all types of data formats (e.g., binary data, log files, text files, etc.) needed to debug defects.

[0022] Debug server 110 can be a centralized server that can simultaneously interact with debug tool 150 and end-user systems 140. Debug server 110 operates intelligent data collection system 120 and maintains dataset metadata 130 for each defect category. This metadata includes predefined dataset content 132 for the defect category and a predefined dataset acquisition order for the defect category. Intelligent data collection system 120 includes a learning component 122 that can interact with debug tool 150 to collect debug data from end-user systems 140 and update defect category metadata 130. Intelligent data collection system 120 can monitor data requests from instances of debug tool 150 for a given defect category and use a learning process to refine the given dataset for that defect category.

[0023] A "defect category" is a type of defect, and a specific dataset is required to debug the type of defect indicated by this defect category. A "core dataset" is the minimum data required to determine the defect category type of a certain defect.

[0024] A "dataset" is a predetermined set of data required to debug defects of a predetermined defect category type. A "full dataset" is the totality of all data available to the system. A "optimal dataset" is the minimum dataset generated based on time to enable debugging. A "dataset order" is the order in which individual data elements in a dataset are most frequently accessed by an engineer when debugging a defect. An "individual data element" may be, for example, a log file, a binary memory dump, or an error log.

[0025] 2A, a flow diagram 200 illustrates an exemplary embodiment of one aspect of the described method for collecting data for defect analysis. The method is performed in intelligent data collection system 120.

[0026] The method provides 201 defect category dataset metadata 130, which includes predefined dataset content 132 and predefined dataset order 134. The dataset metadata is learned through a learning process by monitoring dataset acquisition from the end system during a debug process. Initially, the predefined dataset content 132 may be the entire dataset in the form of all data available to the system, which may be adapted over time to an optimal dataset through a learning process. The learning process may use statistical learning or machine learning, and the learning evolves over time.

[0027] The metadata learning process uses an algorithm within the system that repeatedly examines the dataset contents selected for a given defect category during each debug phase. A "heat map" may be generated of the most used dataset elements (data pieces from the dataset used in debugging). The "most used" dataset resulting from this analysis is periodically matched with the currently recorded dataset for the defect category, and this dataset is adjusted to match the data elements of the most selected dataset.

[0028] Similarly, an algorithm within the system repeatedly examines the order in which dataset elements are selected for a given debug category during each debug phase. A "heat map" may be generated of the most popular selection order. The resulting most popular order is periodically checked against the dataset order for the debug category, and the dataset order is adjusted to match the most popular selection order.

[0029] At 202, when a defect is notified, a data collection process may be initiated. The method may access a core dataset from the end-user system 140 at 203. The core dataset is the minimum data required to classify the defect into a defect category type. The method may analyze the core dataset at 204 to classify the notified defect into a defined defect category. This classification may involve comparing aspects of the notified defect to the core dataset.

[0030] The method may obtain 205 current dataset metadata for a defined defect category for the notified defect. The current dataset metadata defines, for this defect category, an up-to-date description of the defined content for the dataset and the defined acquisition order for the dataset. The dataset definitions may be stored in the intelligent data collection system, and each dataset defines which elements of data (e.g., files, logs, etc.) are required to debug the dataset. The intelligent data collection system uses the dataset definition for the debug category to determine which data elements should be collected from the end-user system and in what order to progress the debug.

[0031] At 206, predefined dataset content may be provided from an end system for use by the debugging tool. At 207, the dataset content may be provided as data elements or groups of data elements in a predefined order according to dataset metadata, thereby preloading the dataset from the end user system into the data store for use by the debugging tool.

[0032] The order of data element acquisition ensures that the person performing the debugging task gets the correct information at the right time and in the correct order to effectively debug the defect. For example, if a debug engineer needs to know the error code and memory address from where to get the data needed to debug the next step, it doesn't make sense to first collect a binary memory dump, then collect the error code log and memory map data. It makes more sense to collect them in the reverse order. In other words, the key is to get the information to the debug engineer in the most logical order.

[0033] 2B, a flow diagram 220 illustrates an exemplary embodiment of another aspect of the described method for collecting data for defect analysis. The method is performed in intelligent data collection system 120.

[0034] The method may begin debugging the defect at 221 by running a debug tool and loading the current dataset contents defined in the dataset metadata.

[0035] At 222, during operation of the debug tool for the defect category, the method may monitor and record requests for data, including access to predefined content for the current data set for the defect category and access to additional content required by the debug tool.

[0036] The method may monitor and record the data set access order for the defined content for the defect category and any other additional content at 223. Data set content and data set access may be provided according to the current metadata; however, debugging operations may request additional data content or request data in a different order as the metadata is refined over time.

[0037] The monitoring and recording at 222 and 223 may monitor the frequency of use or access for the dataset for the defect category. The method may also build a record of dataset accesses and dataset ordering for the defect category over time at 224. This may build a record of the number of times a particular element of data is requested by a debug tool for a given defect category, and the order in which individual elements of data within the dataset are requested. This may include requests to access data not currently included in the dataset.

[0038] The method may use a learning process to modify dataset metadata for a given defect category based on the constructed records to generate an optimal dataset at 225. This modification may result in changes to dataset content and dataset ordering.

[0039] The modification of the dataset content may be based on a "most used" algorithm, so that over time the dataset for a given defect category becomes the optimal dataset needed to debug defects in that category.

[0040] The reordering of data sets for a given defect category may be based on a "most used" algorithm, i.e., the most frequently used data access request order, and over time, primary, secondary, and tertiary data groups are formed based on the ordering of data groups requested by the user.

[0041] Continuous analysis of debug data access patterns is used to adjust dataset content and dataset ordering metadata. The dataset content and dataset ordering metadata are not fixed for each code release, but rather are dynamically updated in real time and are code-level independent. An advantage of this model is that all code levels, past and present, can benefit from the debug learning learned by the system over time. This advantage includes the ability to apply this learning retroactively to previous code releases. This learning occurs across end systems deployed in the field, regardless of the code level running on the end system.

[0042] To distinguish between types of defects, defects are classified into defect categories, which require a specific set of data to debug the defect. To classify defect instances, one or more features are required to create distinct categories. Such features may include and / or combine the following: a unique software assert string; an event / error log; a string pattern from a trace file; a set of entries in a stack trace (which is a history of recent processing activity); a definable set of binary data from a memory trace; and other features appropriate to the desired debugging goal. Using one or more of the above features, a unique defect category can be defined. The data elements used to classify the defect category indicate the required content of the core dataset.

[0043] Each defect category is associated with a predetermined data set. When the debug system is first commissioned, i.e., before any learning activities are performed by the debug system, every defect category may have a data set corresponding to the entire data set associated with that category. Over time, and as learning is learned from acquiring data for the debug process, the data set for the category is refined and approaches an optimal state. Each defect category includes defect category metadata, including defined data set contents and a defined data set order. Additional operational metadata associated with a defect category may include a history of data access requests (i.e., what data was requested) and a history of the order in which data access was requested.

[0044] An exemplary system workflow is described below.

[0045] A defect is notified, which triggers an intelligent data gathering system (IDGS) to retrieve a core dataset from the end-user system. The IDGS analyzes the core dataset to determine the defect category.

[0046] Using this information about the defect category, IDGS determines the dataset needed to debug the problem according to the most recent dataset description provided in the dataset metadata for the defect category.

[0047] The IDGS retrieves the datasets for that defect category from the end-user systems. The order in which the data is retrieved from the end-user systems is indicated by the dataset order recorded in the metadata for that defect category. The purpose is to allow engineers to begin debugging work as quickly as possible on the datasets in the most frequently used data analysis order for a given defect category.

[0048] The engineer begins debugging the defect by running the debug tool and loading the data set.

[0049] Each time an engineer accesses a portion of the dataset, the debug tool communicates with IDGS, which records the access request in a data record for the defect category. This is true whether the access request is for data already included in the dataset or if the data is not currently included in the dataset. IDGS also records the order in which the data access requests were made.

[0050] If an engineer requests access to data not currently included in the dataset for a defect category, IDGS will retrieve that data from the end-user system for the engineer to use in analyzing the defect.

[0051] Over time, the IDGS builds a record of the number of times a particular bit of data was requested by a debug tool for a given defect category, and the order in which individual bits of data within a data set were requested.

[0052] IDGS uses learning to modify the dataset for a given defect category based on the "most used" algorithm. As a result, over time, the dataset for a given defect category becomes the optimal dataset needed to debug the type of defects represented by that defect category. Note that data can be added or removed from the dataset based on whether or not the data is used. Data that has not been used for a threshold number of n instances is removed from the optimal dataset.

[0053] IDGS uses learning to reorder data sets for a given defect category based on a "most used" algorithm, i.e., the most frequently used data access request order. Over time, primary, secondary, and tertiary data sets are formed based on the ordering of data groups requested by users.

[0054] Thus, over time, the optimal data sets and the data set order for collecting that data are algorithmically defined by the system for successful and efficient debugging of defects.

[0055] 3A and 3B, schematic block diagrams 300, 340 illustrate optimization of data set content 321 and data set order 322 for collection of data for defect category 320. FIG.

[0056] FIG. 3A shows a series of dates 311-314 during which a defect category is debugged. Each letter represents an element of data within the dataset. Initially, the dataset consists of data A, B, C, and D 331-334. Access requests 321-324 are made on each debug date 311-314. On day 1 311, access request 321 is for data elements A, C, D, and E. On day 2 312, access request 322 is for data elements A, C, D, and E. On day 3 313, access request 323 is for data elements A, B, C, and D. On day 4 314, access request 324 is for data elements A, C, D, and E. Over time, a pattern emerges regarding the most frequently requested data elements. The dataset metadata 321 for defect category 320 is adjusted from the initial dataset of A, B, C, and D to the newly learned, optimized dataset of A, C, D, and E.

[0057] FIG. 3B shows a series of dates 351-354 on which a defect category is debugged. Each letter represents an element of data within the dataset. Initially, dataset sequences 371-374 consist of data elements A, B, C, and D. Access requests 361-364 are made on each debug date 351-354. Access request sequence 361 on day 1 351 is for data elements A, D, C, and B. Access request sequence 362 on day 2 352 is for data elements A, D, C, and B. Access request sequence 363 on day 353 is for data elements A, C, D, and B. Access request sequence 364 on day 4 354 is for data elements A, D, C, and B. Over time, a pattern emerges regarding the most frequently requested sequences. The dataset order metadata 322 for the defect category 320 is adjusted from A, B, C, D in the initial dataset to A, D, C, B, which is the newly learned optimized dataset order.

[0058] 4A and 4B show schematic network map diagrams 400 and 450. These diagrams illustrate how an intelligent data collection system can internally model dataset content and dataset ordering for a first defect category shown in FIG. 4A and a second defect category shown in FIG. 4B. The size of each node represents how frequently a dataset element (A-J) is used to debug the defect category. The pattern of each node represents the most common order in which the dataset elements are selected. For example, nodes with vertical lines represent primary data, nodes with horizontal lines represent secondary data, and nodes with dots represent tertiary data. In FIG. 4A, nodes C, H, and J are depicted with vertical lines representing primary data, nodes A, B, D, E, F, and G are depicted with horizontal lines representing secondary data, and node I is depicted with dots representing tertiary data. In Figure 4B, nodes D, F, and I are depicted with vertical lines representing primary data, nodes A, B, E, J, G, and H are depicted with horizontal lines representing secondary data, and node C is depicted with a dotted pattern representing tertiary data.

[0059] 5, a block diagram illustrates a computing system 500 in which the intelligent data collection system 120 may be provided. The computing system 500 includes at least one processor 501, hardware modules, or circuitry for performing the functions of the described components (which may be software units executing on at least one processor). Multiple processors may be provided that execute parallel processing threads, thereby enabling parallel processing of some or all of the functionality of the components. A memory 502 may be configured to provide computer instructions 503 to the at least one processor 501 to perform the functions of the components of the intelligent data collection system 120.

[0060] The intelligent data collection system 120 may include a metadata generation component 511 that generates, for each defined category, metadata including defined content for the dataset and a defined acquisition order for each element of the dataset, and updates the metadata using the learning component 122.

[0061] The intelligent data collection system 120 may include a category determination component 520 that determines a predefined defect category for the notified defect. The category determination component 520 may include a core dataset component 521 that provides a core dataset for defect category classification, and a classification component 522 that classifies the notified defect by comparing the notified defect to the core dataset. The classification component 522 may compare characteristics of the notified defect with the core dataset.

[0062] The intelligent data collection system 120 may include a metadata acquisition component 513 that acquires current dataset metadata for a predefined defect category of the notified defect, where the dataset metadata includes the predefined contents of the dataset and a predefined acquisition order for each element of the dataset.

[0063] The intelligent data collection system 120 may include a dataset provisioning component 514 that loads datasets from end systems for notified defects for use by the debug tool, the datasets having predefined content and being loaded in a predefined order according to the dataset metadata.

[0064] The intelligent data collection system 120 may include a monitoring component 515 that monitors data set accesses, including accesses to predefined content related to the current data set metadata for the defect category and accesses to additional content required by the debug tool during operation of the debug tool for the defect category. The monitoring component 515 may also be for monitoring the data set access order of the predefined content and the order of access to any other additional content.

[0065] The intelligent data collection system 120 may include a recording component 516 that maintains a record of data set accesses and data set access orders. The learning component 122 may be for applying a learning process, such as statistical learning or machine learning, to update the data set metadata for the defect category based on the recording component 516.

[0066] The intelligent data collection system 120 may include an additional metadata generation component 517 that generates additional operational metadata associated with the defect category, including a history of data access requests and a history of the order in which the data access was requested.

[0067] Various aspects of the present disclosure are described in text, flowcharts, block diagrams of computer systems, and / or block diagrams of machine logic included in computer program product (CPP) embodiments. For any flowchart, depending on the technology involved, operations may be performed in an order different from that shown in a given flowchart. For example, two operations shown in successive blocks of a flowchart may be performed in reverse order, as a single integrated step, simultaneously, or with at least partial time overlap, again depending on the technology involved.

[0068] A computer program product embodiment ("CPP embodiment" or "CPP") is a term used in this disclosure to describe any one or more storage media (also referred to as "media") collectively included in one or more storage devices that collectively contain machine-readable code corresponding to instructions and / or data for performing the computer operations defined in a given CPP claim. A "storage device" is any tangible device capable of holding and storing instructions for use by a computer processor. The computer-readable storage medium may be, but is not limited to, an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, a mechanical storage medium, or any suitable combination of the foregoing media. Some known types of storage devices that include these media include diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanically encoded devices (e.g., punch cards or pits / lands formed on a major surface of a disk), or any suitable combination of the aforementioned devices. Computer-readable storage media, as the term is used in this disclosure, should not be construed as storing signals that are transitory in nature, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide, light pulses passing through fiber optic cables, electrical signals transmitted through wires, and / or other transmission media.As those skilled in the art will appreciate, data is typically moved at some infrequent time during the normal operation of a storage device, for example, during access, defragmentation, or garbage collection, but this does not cause the storage device to be considered temporary, since the data is not temporary while it is stored.

[0069] 6 , computing environment 600 includes an example of an environment for executing at least a portion of the computer code (e.g., intelligent data collection code 650) associated with performing the methodology of the present invention. In addition to block 650, computing environment 600 includes, for example, computer 601, wide area network (WAN) 602, end user device (EUD) 603, remote server 604, public cloud 605, and private cloud 606. In this embodiment, computer 601 includes a set of processors 610 (including processing circuitry 620 and cache 621), a communications fabric 611, volatile memory 612, persistent storage 613 (including operating system 622 and block 650 shown above), a set of peripheral devices 614 (including a set of user interface (UI) devices 623, storage device 624, and a set of Internet of Things (IoT) sensors 625), and a network module 615. Remote server 604 includes a remote database 630. The public cloud 605 includes a gateway 640, a cloud orchestration module 641, a set of host physical machines 642, a set of virtual machines 643, and a set of containers 644.

[0070] Computer 601 may take the form of a desktop computer, laptop computer, tablet computer, smartphone, smartwatch or other wearable computer, mainframe computer, quantum computer, or any other type of computer or mobile device now known or later developed, capable of executing programs, accessing a network, or querying a database, such as remote database 630. As is well understood in the computer technology field and depending on the technology, execution of a computer-implemented method may be distributed across multiple computers and / or multiple locations. While this description of computing environment 600 focuses on and discusses one computer, specifically computer 601, in detail to keep the description as concise as possible. Computer 601 may be located within a cloud, even though it is not depicted within the cloud in FIG. 6 . However, computer 601 is not required to reside within a cloud except to the extent explicitly stated.

[0071] The processor set 610 includes one or more computer processors of any type now known or later developed. The processing circuitry 620 may be distributed across multiple packages, e.g., multiple integrated circuit chips working together. The processing circuitry 620 may implement multiple processor threads and / or multiple processor cores. The cache 621 is memory located within the processor chip package and is typically used for data or code available for fast access by the threads or cores operating in the processor set 610. Cache memory is typically divided into multiple levels depending on relative proximity to the processing circuitry. Alternatively, some or all of the cache for a processor set may be located "off-chip." In some computing environments, the processor set 610 may be designed to operate with qubits and perform quantum computing.

[0072] Computer-readable program instructions are typically loaded into computer 601 and cause processor set 610 of computer 601 to perform a series of operational steps, thereby performing a computer-implemented method. As a result, the instructions so executed instantiate the methods defined in the computer-implemented method flowcharts and / or descriptions contained herein (collectively, the "methods of the present invention"). These computer-readable program instructions are stored in various types of computer-readable storage media, such as cache 621 and other storage media described below. The program instructions and associated data are accessed by processor set 610 to control and direct the execution of the methods of the present invention. In computing environment 600, at least a portion of the instructions for performing the methods of the present invention may be stored in block 650 of persistent storage 613.

[0073] Communications fabric 611 is the signal transmission pathway that allows the various components of computer 601 to communicate with one another. Typically, this fabric is made up of switches and conductive pathways that form, for example, buses, bridges, physical input / output ports, etc. Other types of signal communication pathways may also be used, for example, fiber optic communication pathways and / or wireless communication pathways.

[0074] Volatile memory 612 may be any type of volatile memory now known or later developed. Examples include dynamic random access memory (RAM) or static RAM. Typically, volatile memory 612 is characterized by random access, although this is not required unless explicitly stated. In computer 601, volatile memory 612 is located in a single package and is internal to computer 601; however, alternatively or additionally, volatile memory may be distributed across multiple packages and / or located external to computer 601.

[0075] Persistent storage 613 is any form of non-volatile storage for a computer, now known or later developed. The term non-volatile storage refers to the fact that stored data is maintained regardless of whether power is supplied to computer 601 and / or directly to persistent storage 613. While persistent storage 613 may be read-only memory (ROM), typically at least a portion of persistent storage allows data to be written to it, data to be erased, and data to be rewritten. Some well-known forms of persistent storage include magnetic disks and solid-state storage devices. Operating system 622 may take several forms, including various known proprietary operating systems or open-source POSIX (Portable Operating System Interface)-type operating systems that use a kernel. The code contained in block 650 typically includes at least some of the computer code associated with performing the methods of the present invention.

[0076] Peripheral device set 614 includes peripheral devices of computer 601. Data communication connections between peripheral devices and other components of computer 601 can be realized in various ways, such as Bluetooth® connections, Near-Field Communication (NFC) connections, connections made via cables (such as universal serial bus (USB)-type cables), pluggable connections (e.g., secure digital (SD) cards), connections made over local area communication networks, and even connections made over wide area networks such as the Internet. In various embodiments, UI device set 623 can include components such as display screens, speakers, microphones, wearable devices (such as goggles and smartwatches), keyboards, mice, printers, touchpads, game controllers, and haptic devices. Storage 624 is an external storage device such as an external hard drive or an insertable storage device such as an SD card. Storage 624 can be persistent and / or volatile. In some embodiments, storage 624 may take the form of a quantum computing storage device that stores data as qubits. In embodiments where computer 601 requires mass storage (e.g., computer 601 stores and manages large databases locally), this storage may be provided by peripheral storage devices designed to store vast amounts of data, such as a storage area network (SAN) shared by multiple geographically distributed computers. IoT sensor set 625 consists of sensors that can be used in Internet of Things applications. For example, one sensor may be a thermometer and another may be a motion detector.

[0077] Network module 615 is a collection of computer software, hardware, and firmware that enables computer 601 to interact with other computers over WAN 602. Network module 615 may include hardware such as a modem or Wi-Fi® signal transceiver, software for packetizing and / or depacketizing data for communication network transmission, and / or web browser software for communicating data over the Internet. In some embodiments, the network control and network forwarding functions of network module 615 are performed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing Software-Defined Networking (SDN)), the control and forwarding functions of network module 615 are performed on physically separate devices, resulting in the control function managing multiple different network hardware devices. Computer-readable program instructions for carrying out the methods of the present invention can typically be downloaded to computer 601 from an external computer or external storage device through a network adapter card or network interface included in network module 615.

[0078] WAN 602 is any wide area network (e.g., the Internet) capable of communicating computer data over long distances by any now known or later developed technology for communicating computer data. In some embodiments, WAN 602 may be replaced and / or supplemented by a local area network (LAN) designed to communicate data between devices located in a local area, such as a Wi-Fi network. WANs and / or LANs typically include copper transmission cables, optical fiber transmissions, wireless transmissions, and computer hardware such as routers, firewalls, switches, gateway computers, and edge servers.

[0079] End-user device (EUD) 603 is any computer system used and managed by an end user (e.g., a customer of the enterprise that operates computer 601) and can take any of the forms described above with respect to computer 601. EUD 603 typically receives useful and actionable data from the operation of computer 601. For example, in the hypothetical case where computer 601 is designed to make recommendations to the end user, this recommendation would typically be communicated from network module 615 of computer 601 over WAN 602 to EUD 603. In this manner, EUD 603 can display or otherwise present the recommendation to the end user. In some embodiments, EUD 603 can be a client device, such as, for example, a thin client, a heavy client, a mainframe computer, a desktop computer, or the like.

[0080] Remote server 604 is any computer system that provides at least some data and / or functionality to computer 601. Remote server 604 may be managed and used by the same business entity that operates computer 601. Remote server 604 represents a machine that collects and stores useful data for use by other computers, such as computer 601. For example, in the hypothetical case where computer 601 is designed and programmed to make recommendations based on historical data, this historical data may be provided to computer 601 from remote database 630 of remote server 604.

[0081] A public cloud 605 is any computer system available for use by multiple business entities that provides on-demand availability of computer system resources and / or other computer functions, particularly data storage (cloud storage) and computing power, without requiring direct user management. Cloud computing typically leverages resource sharing to achieve consistency and economies of scale. Direct and autonomous management of the computing resources of the public cloud 605 is performed by computer hardware and / or software in a cloud orchestration module 641. The computing resources provided by the public cloud 605 are typically implemented by virtual computing environments running on various computers comprising a host physical machine set 642, which refers to all physical computers present in and / or available to the public cloud 605. Virtual computing environments (VCEs) typically take the form of virtual machines from a virtual machine set 643 and / or containers from a container set 644. It is understood that these VCEs may be stored as images and transferred among and between various hosts of physical machines as images or after instantiation of the VCE. Cloud orchestration module 641 manages the transfer and storage of images, deploys new instances of VCEs, and manages active instances from VCE deployments. Gateway 640 is a collection of computer software, hardware, and firmware that enables public cloud 605 to interact over WAN 602.

[0082] Here, we provide some further explanation of virtual computing environments (VCEs). A VCE can be stored as an "image." A new, active instance of a VCE can be instantiated from that image. Two well-known types of VCEs are virtual machines and containers. A container is a VCE that uses operating system-level virtualization. This refers to a feature of an operating system in which the kernel allows the existence of multiple isolated user space instances, called containers. These isolated user space instances typically behave as actual computers from the perspective of the programs running within them. A computer program running on a typical operating system can utilize all of the computer's resources, such as connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, a program running inside a container can only use the contents of the container and the devices assigned to the container; this feature is known as containerization.

[0083] Private cloud 606 is similar to public cloud 605, except that its computing resources are available only for use by a single enterprise. While private cloud 606 is depicted as interacting with WAN 602 in other embodiments, a private cloud may be completely disconnected from the Internet and accessible only through a local / private network. A hybrid cloud is a composite of multiple clouds of different types (e.g., private, community, or public cloud types), each often implemented by a different vendor. While each of the multiple clouds remains a separate, isolated entity, the larger hybrid cloud architecture is unified by standardized or proprietary technologies to enable orchestration, management, and / or data / application portability between the constituent clouds. In this embodiment, both public cloud 605 and private cloud 606 are part of a larger hybrid cloud.

[0084] The description of various embodiments of the present invention has been presented for illustrative purposes, but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terms used herein have been selected to best explain the principles, practical applications, or technical improvements of the embodiments to technology found in the market, or to enable those skilled in the art to understand the embodiments disclosed herein.

[0085] Improvements and modifications can be made to the above-described embodiments without departing from the scope of the present invention.

Claims

1. 1. A computer-implemented method for optimizing data collection for defect analysis, comprising: determining a predefined defect category for the notified defect; acquiring current dataset metadata for the defined defect category, wherein the dataset metadata includes predefined content for the dataset and a predefined acquisition order for elements of the dataset, wherein the dataset metadata is learned by monitoring dataset acquisition from an end system during a debug process; and providing a dataset from an end system for use by a debugging tool, wherein the dataset has the defined content and is loaded in the defined order according to the dataset metadata. A method comprising:

2. generating, using a learning process, metadata for each predefined category, the metadata including predefined content for the dataset and a predefined retrieval order for elements of the dataset; The method of claim 1 further comprising:

3. The step of determining a predefined defect category for the notified defect further comprises: providing a core data set for defect categorization; and classifying the notified defect by comparing the notified defect to the core data set.

2. The method of claim 1, comprising:

4. The method of claim 3 , wherein comparing the notified defect to the core dataset includes comparing characteristics of the notified defect to the core dataset.

5. monitoring dataset accesses, including accesses to the predefined content related to the current dataset metadata for the defect category and accesses to additional content required by the debugging tool during operation of the debugging tool for the defect category; maintaining a record of dataset access; and applying a learning process to update the dataset metadata for the defect category. The method of claim 1 further comprising:

6. applying the learning process to update the dataset metadata further comprises: Deleting dataset content that has not been accessed for a predetermined number of defect category instances.

6. The method of claim 5, comprising:

7. monitoring a data set access sequence for said defined content and a sequence of access to any other additional content; maintaining a record of the data set access order; and applying a learning process to update dataset metadata for the defect category. The method of claim 1 , further comprising:

8. The method of claim 1 , wherein an initial dataset metadata for a defect category defines all the content of the dataset and is reduced over time to an optimal dataset.

9. The method of claim 1 , wherein the predefined acquisition order for elements of the data set includes elements of individual data stored in a data log or memory dump.

10. generating additional operational metadata associated with the defect categories, the metadata including a history of data access requests and a history of the order in which the data access was requested; The method of claim 1 , further comprising:

11. 1. A computer system for optimizing data collection for defect analysis, comprising: a processor and a memory configured to provide said processor with computer program instructions for performing the functions of each component; a category determination component that determines a predefined defect category for the notified defect; a metadata acquisition component that acquires current dataset metadata for the defined defect category, where the dataset metadata includes predefined content for the dataset and a predefined acquisition order for elements of the dataset, where the dataset metadata is learned by monitoring dataset acquisition from an end system during a debug process; and a dataset providing component that provides a dataset from an end system for use by a debugging tool, wherein the dataset has the defined content and is loaded in the defined order according to the dataset metadata; A computer system comprising:

12. 12. The computer system of claim 11, further comprising a metadata generation component that generates and updates metadata for each defined category using a learning process, the metadata including predefined content for the dataset and a predefined retrieval order for elements of the dataset.

13. The category determination component: A core dataset component that provides a core dataset for defect categorization; and a classification component that classifies the notified defect by comparing the notified defect to the core dataset; 12. The computer system of claim 11, comprising:

14. The computer system of claim 13 , wherein the classification component compares characteristics of the notified defect to the core data set.

15. a monitoring component that monitors dataset access during operation of a debug tool for a defect category, including access to the defined content related to the current dataset metadata for the defect category and access to additional content required by the debug tool; a records component that maintains records of dataset accesses; and a learning component that applies a learning process to update the dataset metadata for the defect categories. The computer system of claim 11 further comprising:

16. 16. The computer system of claim 15, wherein the learning component is further to apply a learning process to update the dataset metadata to remove dataset content that has not been accessed for a predetermined number of defect category instances.

17. 16. The computer system of claim 15, wherein the monitoring component is for monitoring a dataset access order for the defined content and an order of access to any other additional content, the recording component is for maintaining a record of the dataset access order, and the learning component is for applying a learning process to update dataset metadata for the defect category.

18. 18. The computer system of claim 11, wherein the predefined acquisition order for elements of the data set includes elements of individual data stored in a data log or memory dump.

19. and an additional metadata generation component that generates additional operational metadata associated with the defect categories, the additional operational metadata including a history of data access requests and a history of the order in which the data access was requested.

18. A computer system according to any one of claims 11 to 17.

20. 1. A computer program for optimizing data collection for defect analysis, comprising: program instructions that, when executed by a processor, cause the processor to: Procedures for determining the defined defect category for the notified defect; acquiring current dataset metadata for the defined defect category, wherein the dataset metadata includes predefined content for the dataset and a predefined acquisition order for elements of the dataset, wherein the dataset metadata is learned by monitoring dataset acquisition from an end system during a debug process; and providing a dataset from an end system for use by a debugging tool, wherein the dataset has the defined content and is loaded in the defined order according to the dataset metadata; A computer program that executes

Images