Method for generating metadata from collected signals for search, filtering and machine learning input

Abstract

The present disclosure relates to test and measurement systems, and more particularly, to automatically generating metadata from acquired signals. A computing device includes a port that allows the computing device to connect to a network, and one or more processors configured to execute code to cause the one or more processors to determine that a new waveform has been added to a repository connected to the computing device, perform a set of measurements on the new waveform, attach results from the measurements to the new waveform as metadata, and store the new waveform and attached metadata to the repository. A method of managing waveform data includes determining that a new waveform has been added to a repository, performing a set of measurements on the new waveform, attaching results from the measurements to the new waveform as metadata, and storing the new waveform and attached metadata to the repository.

Description

[0001] Related applications

[0002] This disclosure claims the benefit of U.S. Provisional Patent Application 63 / 147,153, filed February 8, 2021, entitled “METHODOF GENERATING METADATA FROM ACQUIRED SIGNALS FOR SEARCH, FILTERING AND MACHINE LEARNING INPUTS”, which is incorporated herein in its entirety. Technical Field

[0003] This disclosure relates to test and measurement systems, and more particularly, to the automatic generation of metadata from acquired signals. Background Technology

[0004] Engineers perform measurements on test and measurement equipment to extract tangible information from the collected data. Information is hidden within waveforms and engineering data, but it is difficult to extract. Unlike searching text—where computers can handle data formats identical to the search format—humans must use text and context to search for and understand waveforms and engineering data.

[0005] This presents a challenge because waveform data must be abstracted into something tangible for the system in order to effectively translate human needs into waveform characteristics or measurements. Furthermore, engineers often need to know what measurements or checks are required at the time of acquisition to solve a specific problem. This results in extracting only the specific information needed to solve a particular problem from the acquired data, rather than other information they might subsequently need as their understanding changes during the debugging / design cycle.

[0006] As an example, suppose a user wants to sort ten waveforms from minimum to maximum root mean square (RMS) values—a common measurement performed on test and measurement equipment such as an oscilloscope. If all files included this measurement, the sorting would be a simple operation. However, suppose the engineer who acquired the candidate waveforms only happened to perform RMS measurements on five of the stored waveforms. In one solution, the system could open each file, perform the missing RMS measurements, and then perform the sorting. However, these files tend to be large, and opening, measuring, and then saving the RMS measurements would take a very long time, making it unscalable to multiple files. Summary of the Invention

[0007] The embodiments of the disclosed apparatus and methods address the shortcomings of the prior art. Attached Figure Description

[0008] Figure 1A system for collecting, analyzing, and storing waveform data and associated metadata is shown;

[0009] Figure 2 The machine learning components of a system for collecting, analyzing, and storing waveform metadata are shown.

[0010] Figure 3 A flowchart illustrating the method for capturing waveform data and metadata is shown;

[0011] Figure 4 A flowchart illustrating the method for analyzing waveform data and metadata is shown. Detailed Implementation

[0012] The implementation design attaches measurements as metadata to waveform files. The challenge lies in determining which measurements to include and how to include them to make the system as useful as possible. The implementation creates libraries of measurements, classifications, decodings, and other algorithms, all of which can be run against the dataset whenever new data is added. As used herein, the term "measurement" refers to any measurement, query, search, filter, classification, decoding, etc., run against data. In addition to the library of measurements, the system also implements methods for automatically fetching historical data already existing in the library. The library can also be updated when new measurements are run to add them to the library.

[0013] The embodiments also provide for expanded uses of the data. As used herein, the term "data" refers to waveform data acquired from test and measurement equipment and may include data points, images, etc. The term "metadata" as used herein refers to any other information attached to the waveform data, and may not actually include the data from the test and measurement equipment. This may include administrative information such as the names of the personnel who performed the test, their contact information, the date and time of the test, and a description of the device under test that generated the waveform, including its serial number. In the embodiments described herein, metadata may also include measurements performed on the data, the results of those measurements, etc.

[0014] For example, in the example given above, the user wants to sort 10 waveforms from smallest to largest RMS value; these measurements will be performed as the data is added to the repository. This will allow the user to perform the classification, as the waveform data will have readily available information.

[0015] For our example, a better solution would be to ensure that RMS measurements are always attached to every file in the system, use automated processes to capture data, and combine this with the virtually unlimited computing resources of a modern cloud environment.

[0016] Figure 1An embodiment of system 10 providing these capabilities to users is illustrated. The system includes computing devices 12, which can effectively include a “cloud” of multiple computing devices such as 14. Each computing device 14 may have one or more processors 16, memory 18, a user interface 20, and a port 22, which allows the device to connect to other assets via a connection 26 to a network 28. Connection 26 may include connections via the Internet, intranets, virtual private networks, or even direct connections. Similarly, a storage vault 24 may be directly connected to computing devices 12, may be part of computing devices 12, or may be connected via network 28.

[0017] As described above, the computing device and the storage library receive waveform data from a test and measurement device, such as 30. The test and measurement device performs tests on the device under test (DUT) 32, thereby generating waveform data used in the system. As will be discussed in more detail later, a user can access the storage library 24 directly or via the computing device 12 using a user device 34 connected to the network 28 via connection 36. The user device 34 may include a computing device, such as a desktop or laptop computer, tablet computer, mobile phone, augmented reality (AR) or virtual reality (VR) headset, or any other computing device that allows the user to access the storage library and / or send signals to the computing device 12 via the network 28. The user device 34 may include a display for presenting the results returned by the computing device 12 to the user.

[0018] like Figure 2 As shown, computing device 12 may include one or more processors programmed into a machine learning system 38. The term "machine learning system" as used herein refers to a system capable of operating autonomously and learning and improving from experience without explicit programming. The programming mentioned above refers to programming the processors to form a machine learning system. The machine learning system will gain experience by consuming a specific dataset (here referred to as the training set). Embodiments of this system may involve supplementing the training set by identifying new datasets, which will be discussed in more detail below. Similar to the discussion above regarding distributed computing devices, storage 24 may include multiple storage units, partitions of a single storage unit, etc. One storage unit 40 may include the library of measurements described above. Another storage unit 42 may contain actual data and metadata. Another storage unit 44 may include a library of training datasets.

[0019] The implementation initially populated the repository with historical data. This historical data could have ultimately witnessed all the measurements applied in the system, such as during periods when the system was not under heavy load. Regarding new data, Figure 3The method for adding new data is illustrated. At 50, the computing device uses one or more processors to determine that a new waveform has been added to the storage. This can be achieved through a notification from the storage or a query to the storage. Then, at 52, the one or more processors perform a measurement set on the waveform.

[0020] The set of measurements performed can include various measurements. One example could include all possible and actual measurements, since the system has little or no knowledge of what measurements need to be performed. Another example could be the execution of a user-defined set of measurements, such as a user-selected power ripple measurement.

[0021] Another example involves executing a defined set of measurements based on an initial evaluation of the signal. The system decides to run automatically, for example, a power ripple measurement based on the characteristics of the input signal. The characteristics of the signal can be determined through an initial evaluation. This evaluation can be based on one or more of the following: classification performed by a machine learning system, pattern recognition, measurement results, user-defined characteristics, and tags / metadata added during acquisition. An example of measurement results could be that if the center frequency is measured to be 915 MHz, then all Z-Wave measurements are run. User-defined characteristics could include things like whether the data contains 200 data points or whether the average amplitude is greater than 3 volts. Examples enabled by the system would involve tags / metadata added during acquisition, such as time / date, operator name, user-added tags (e.g., "USB"), etc. Another example could be based on the status of other systems, such as "Run certain measurements based on health reports of other systems in the lab."

[0022] Back Figure 3 Once the system measures the waveform at step 52, the result is attached to the waveform data at step 54. Then, at step 56, this is stored in a repository. As mentioned above, at step 58, the data can be added to the training dataset for future updates to the machine learning system. This determination can be made by comparing the new data with the existing data to determine which attributes the existing data does not possess, and thus adding them to the training set. In another analysis, the machine learning system can have an error vector, and the dataset can be selected to improve the error vector.

[0023] With this system in place, addressable use cases for automatic metadata creation can be extended. Figure 4 An embodiment of the method using the system is shown. At 60, the computing device... Figure 1The user equipment shown receives user input. The user input at 62 submits a request to the computing device to perform a task, and this task can take one of several forms. Searching allows new searches to be applied to existing data, such as "finding the I2C decoder using address = x40h". Each measurement or algorithm can create structured data like numbers and strings, which have sortable properties, such as waveform searches in ascending order of RMS values. When waveforms are refined into measurements, complex filters can be written for searching and automation, such as "showing digital waveforms with rising edges between 100 μs and 1 ms".

[0024] Using machine learning or other methods, the system can categorize waveforms and measurements into highly searchable and human-readable types, such as "show all CAN bus captures". Users can create notifications from metadata / measurements, such as "send an email and link to the user when the process finds xAF, xFF, x16, x45 on RS-232 signals". Users can also request the system to detect certain things, features, trends, thresholds, errors, etc., such as "show all detected ultrashort frames (runts)".

[0025] One advantage of machine learning systems is their ability to make predictions. Machine learning systems can make predictions about future measurements, such as "showing all waveforms indicating a 95% probability of failure."

[0026] The computing device then presents the results to the user at 64. At 66, the user inputs, such as the search or query discussed above, which can then be saved to a library of measurements, etc.

[0027] In this way, as new measurements or capabilities are added to the system, new metadata can be continuously collected and aggregated. New metadata on new data entering the system can also be collected and aggregated, allowing for the searching and utilization of the new data for other updated tasks.

[0028] Various aspects of this disclosure can operate on specially created hardware, on firmware, on a digital signal processor, or on a specially programmed general-purpose computer including a processor that operates according to programmed instructions. The terms controller or processor used herein are intended to include microprocessors, microcomputers, application-specific integrated circuits (ASICs), and special-purpose hardware controllers. One or more aspects of this disclosure can be embodied in computer-usable data and computer-executable instructions, such as one or more program modules executed by one or more computers (including monitoring modules) or other devices. Typically, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types when executed by a processor in a computer or other device. Computer-executable instructions can be stored on non-transitory computer-readable media, such as hard disks, optical disks, removable storage media, solid-state storage, random access memory (RAM), etc. As those skilled in the art will appreciate, the functionality of program modules can be combined or distributed in various aspects as desired. Furthermore, functionality can be wholly or partially embodied in firmware or hardware equivalents, such as integrated circuits, FPGAs, etc. Specific data structures can be used to more efficiently implement one or more aspects of this disclosure, and such data structures are envisioned within the scope of the computer-executable instructions and computer-available data described herein.

[0029] In some cases, the disclosed aspects may be implemented in hardware, firmware, software, or any combination thereof. The disclosed aspects may also be implemented as instructions carried on or stored on one or more non-transitory computer-readable media, which may be read and executed by one or more processors. Such instructions may be referred to as a computer program product. As discussed herein, a computer-readable medium means any medium that can be accessed by a computing device. By way of example and not limitation, a computer-readable medium may include computer storage media and communication media.

[0030] Computer storage media refers to any medium that can be used to store computer-readable information. By way of example and not limitation, computer storage media may include RAM, ROM, electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital video disc (DVD) or other optical disc storage devices, cassette tape, magnetic tape, disk storage devices or other magnetic storage devices, and any other volatile or non-volatile, removable or non-removable medium implemented in any technology. Computer storage media does not include the signal itself or the temporary form of signal transmission.

[0031] A communication medium is any medium that can be used for the communication of computer-readable information. By way of example and not limitation, a communication medium may include coaxial cable, fiber optic cable, air, or any other medium suitable for communication of electrical, optical, radio frequency (RF), infrared, acoustic, or other types of signals.

[0032] Furthermore, this written description refers to specific features. It should be understood that the disclosure in this specification includes all possible combinations of those specific features. For example, where a specific feature is disclosed in the context of a particular aspect, that feature may also be used in the context of other aspects to the extent possible.

[0033] Example

[0034] Illustrative examples of the disclosed technology are provided below. Embodiments of the technology may include one or more examples described below, as well as any combination of these examples.

[0035] Example 1 is a computing device, including: a port allowing the computing device to connect to a network; and one or more processors configured to execute code such that the one or more processors: determine that a new waveform has been added to a storage library connected to the computing device; perform a set of measurements on the new waveform; attach the results from the measurements to the new waveform as metadata; and store the new waveform and the attached metadata in the storage library.

[0036] Example 2 is a computing device of Example 1, wherein the one or more processors are further configured to execute code such that the one or more processors: receive a new set of measurements from a user device across a network; and add the new set of measurements to a storage library.

[0037] Example 3 is a computing device of Example 2, wherein the one or more processors are further configured to execute code to cause the one or more processors to: perform a new set of measurements on existing waveforms in a storage library; and add the results from the new set of measurements to the metadata attached to each waveform.

[0038] Example 4 is a computing device of any one of Examples 1 to 3, wherein the one or more processors are further configured to execute code to cause the one or more processors to: receive a signal from a user device; perform at least one task indicated by the signal; and provide the result of the task to the user device.

[0039] Example 5 is the computing device of Example 4, wherein the at least one task includes one or more of searching, sorting, filtering, classifying, triggering, detecting, and predicting.

[0040] Example 6 is a computing device of any of Examples 1 to 5, wherein at least one of the one or more processors is configured as part of a machine learning system.

[0041] Example 7 is the computing device of Example 6, wherein the machine learning system is configured to classify waveforms in a storage library into waveform types.

[0042] Example 8 is the computing device of Example 6, in which a machine learning system is configured to provide predictions about future measurements.

[0043] Example 9 is a computing device of any one of Examples 1 to 8, wherein the code that causes the one or more processors to execute a measurement set includes code that causes the one or more processors to execute one or more of the following: all available measurements; a user-definable measurement set; and a defined measurement set after an initial evaluation.

[0044] Example 10 is a computing device of Example 9, wherein the code that causes the one or more processors to perform an initial evaluation includes code that causes the one or more processors to perform at least one of the following: operating a machine learning process to produce a classification; performing pattern recognition; and performing an initial evaluation based on one of the measurement results, user-defined features, and metadata attached to the waveform.

[0045] Example 11 is a computing device of any of Examples 1 to 10, wherein the computing device is connected to one or more test and measurement devices.

[0046] Example 12 is a computing device of any one of Examples 1 to 11, wherein the computing device includes a plurality of computing devices and the one or more processors are distributed among the plurality of computing devices.

[0047] Example 13 is a method for managing waveform data, comprising: determining that a new waveform has been added to a repository; performing a set of measurements on the new waveform; attaching the results from the measurements to the new waveform as metadata; and storing the new waveform and the attached metadata in a repository.

[0048] Example 14 is the method of Example 13, further including: receiving a new measurement set from a user device; and adding the new measurement set to a storage library.

[0049] Example 15 is a method of Example 14, further comprising: performing a new set of measurements on existing waveforms in the repository; and adding the results from the new set of measurements to the metadata attached to each waveform.

[0050] Example 16 is a method of any one of Examples 13 to 15, further comprising: receiving a signal from a user equipment; performing at least one task indicated by the signal; and providing the result of the task to the user equipment.

[0051] Example 17 is a method of Example 16, wherein the at least one task includes one or more of searching, sorting, filtering, classifying, triggering, detecting, and predicting.

[0052] Example 18 is a method of any of Examples 13 through 17, further including using machine learning to classify waveforms in the repository into waveform types.

[0053] Example 19 is a method of any of Examples 13 through 17, and further includes using machine learning to provide predictions about future measurements of the waveform based on metadata.

[0054] Example 20 is a method of any of Examples 13 to 19, wherein executing a measurement set includes executing at least one of all available measurements, a user-definable measurement set, and a defined measurement set after the initial evaluation.

[0055] Although specific embodiments have been illustrated and described for illustrative purposes, it should be understood that various modifications may be made without departing from the spirit and scope of this disclosure. Therefore, the invention should not be limited except for the appended claims.

Claims

1. A computing device, comprising: Ports that allow computing devices to connect to the network; and One or more processors, the one or more processors being configured to execute code to cause the one or more processors to: It was determined that the new waveform had been added to the storage library connected to the computing device; Perform a measurement set on the new waveform; The results from the measurement are attached to the new waveform as metadata, which also includes management information related to the acquisition and test set results of the new waveform; Store the new waveform and attached metadata in the repository; The library of measurements and algorithms is updated based on the set of measurements performed on the new waveform, wherein the library is configured to be dynamically updated to include new measurements and algorithms. and Historical waveform data stored in the repository is retrieved to apply updated measurement and algorithm libraries, thereby attaching updated metadata to the historical waveform data.

2. The computing device of claim 1, wherein the one or more processors are further configured to execute code to cause the one or more processors to: Receive new measurement sets from user equipment across the network; and Add the new measurement set to the library.

3. The computing device of claim 2, wherein the one or more processors are further configured to execute code to cause the one or more processors to: Perform a new set of measurements on existing waveforms in the storage library; and The results from the new measurement set are added to the metadata attached to each waveform.

4. The computing device of claim 1, wherein the one or more processors are further configured to execute code to cause the one or more processors to: Receive signals from user equipment; Perform at least one task indicated by a signal; and Provide the results of the task to the user equipment.

5. The computing device of claim 4, wherein the at least one task includes one or more of searching, sorting, filtering, classifying, triggering, detecting, and predicting.

6. The computing device of claim 1, wherein at least one of the one or more processors is configured as part of a machine learning system.

7. The computing device of claim 6, wherein the machine learning system is configured to classify waveforms in a storage library into waveform types.

8. The computing device of claim 6, wherein the machine learning system is configured to provide predictions about future measurements.

9. The computing device of claim 1, wherein the code causing the one or more processors to execute the measurement set includes code causing the one or more processors to execute one or more of the following: all available measurements; a user-definable measurement set; and a defined measurement set after an initial evaluation.

10. The computing device of claim 9, wherein the code causing the one or more processors to perform an initial evaluation includes code causing the one or more processors to perform at least one of: operating a machine learning process to generate a classification; performing pattern recognition; and performing an initial evaluation based on one of measurement results, user-defined characteristics, and metadata attached to the waveform.

11. The computing device of claim 1, wherein the computing device is connected to one or more test and measurement devices.

12. The computing device of claim 1, wherein the computing device comprises a plurality of computing devices, and the one or more processors are distributed among the plurality of computing devices.

13. A method for managing waveform data, comprising: It has been confirmed that the new waveform has been added to the storage library; Perform a measurement set on the new waveform; The results from the measurement are attached to the new waveform as metadata, which also includes management information related to the acquisition and test set results of the new waveform; Store the new waveform and attached metadata in the repository; The library of measurements and algorithms is updated based on the set of measurements performed on the new waveform, wherein the library is configured to be dynamically updated to include new measurements and algorithms. and Historical waveform data stored in the repository is retrieved to apply updated measurement and algorithm libraries, thereby attaching updated metadata to the historical waveform data.

14. The method of claim 13, further comprising: Receive new measurement sets from user equipment; and Add the new measurement set to the library.

15. The method of claim 14, further comprising: Perform a new set of measurements on existing waveforms in the storage library; and The results from the new measurement set are added to the metadata attached to each waveform.

16. The method of claim 13, further comprising: Receive signals from user equipment; Perform at least one task indicated by a signal; and Provide the results of the task to the user equipment.

17. The method of claim 16, wherein the at least one task includes one or more of searching, sorting, filtering, classifying, triggering, detecting, and predicting.

18. The method of claim 13, further comprising using machine learning to classify waveforms in the repository into waveform types.

19. The method of claim 13, further comprising using machine learning to provide predictions about future measurements of the waveform based on metadata.

20. The method of claim 13, wherein executing the measurement set includes executing at least one of all available measurements, a user-definable measurement set, and a defined measurement set after the initial evaluation.

Images