Artificial intelligence waveform assistant

Abstract

A computing device includes one or more memories, the memories include test and measurement knowledge, a generative artificial intelligence (AI) model accessible to the one or more memories, a display, a user control device allowing a user to provide input, and an application programming interface (API) configured to perform access of the one or more processors to an AI assistant for the generative AI model The method includes generating a generative AI model to allow a user to interact with the AI assistant, receiving one or more user inputs through one or more of the API or user interface, the user inputs providing a description of one or more waveforms to be generated, customizing each waveform definition according to the description using the AI assistant and accessing the generative AI model, and generating the generated AI model based on the customized waveform definition. The one or more waveforms are received and stored.

Description

[0001] Cross-reference with related applications

[0002] This disclosure is a non-provisional content of and claims the benefit thereof in U.S. Provisional Application No. 63 / 649204 entitled “Artificial Intelligence Waveform Assistant”, filed on May 17, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This invention relates to test and measurement instruments, and more particularly, to artificial intelligence agents for test and measurement instruments. Background Technology

[0004] Using machine learning to test and measure the performance of electronic devices has significantly accelerated and improved design testing and verification, as well as production line manufacturing. However, some machine learning environments can present challenges due to a lack of large, standards-related datasets, varying signal impairments used for testing, and different instruments. The lack of datasets is particularly problematic for emerging standards where hardware is unavailable or in short supply. Attached Figure Description

[0005] Figure 1 An example of a system for generating waveforms for a test system is shown.

[0006] Figure 2 A flowchart illustrating an embodiment of a method for generating waveforms for a test system is shown.

[0007] Figure 3 A flowchart of an alternative embodiment of a method for generating waveforms for a test system is shown. Detailed Implementation

[0008] The embodiments described herein involve using generative artificial intelligence (AI) models, such as large language models (LLMs), to create waveform files by combining knowledge and functionality related to standards, damage, and instruments. In some examples, this disclosure demonstrates the use of generative AI models to create tensors.

[0009] Some machine learning systems and processes use tensors as input to the machine learning system. A tensor includes one or more waveform images generated by the device under test (DUT). Tensors may include bar charts and other data. Examples of tensors are disclosed in U.S. Patent Application 17 / 747954, filed April 18, 2022, entitled “SHORT PATTERN WAVEFORM DATABASE BASEDMACHINE LEARNING FOR MEASUREMENT”, U.S. Patent Application 18 / 510234, filed May 23, 2024, entitled “METHODS FOR 3D TENSOR BUILDER FOR INPUT TO MACHINE LEARNING”, and U.S. Patent Application 18 / 665258, filed November 28, 2024, entitled “USER INTERFACE FOR A TENSOR BUILDER TO CONSTRUCTIMAGES FOR INPUT TO MACHINE LEARNING”.

[0010] Some generative AI model providers offer users large language models (LLMs), and some have begun offering AI assistant extensions. AI assistant extensions include several application programming interfaces (APIs) that allow users to access further capabilities. These capabilities include hints and documentation that add to the AI ​​assistant's knowledge, as well as functionality that allows the AI ​​assistant to interact with local or online resources. The ability to interact with resources forms a bidirectional connection, expecting not only actions but also responses—a relatively new feature. Users have the ability to add documentation that can extend the AI ​​assistant's knowledge. The APIs now allow for Retrieval Augmentation (RAG), which allows AI models to be augmented with domain-specific information.

[0011] Generative AI models undergo a training process where the model learns underlying patterns in training data and uses this training to generate new data based on input, which in this embodiment comes from the user of the AI ​​assistant. RAGs differ from training in that they may originate from information sought by the generative AI model itself. RAGs provide the generative AI model with domain-specific data and information, including updates. The generative AI model retrieves information before providing a response. This system may include resources within a firewall or otherwise controlled to allow the generative AI model access.

[0012] Furthermore, the system can employ a vector database to store information using embeddings. While vector databases are a key component of the RAG system, they represent different technologies. Vector databases enable semantic search by representing data such as images, graphics, text, and numerical values ​​as vectors that can be compared in similarity. In RAG, embeddings are retrieved from the vector database and used to enhance the language model with domain-specific knowledge.

[0013] The API features are sufficient for users to describe a large set of waveform characteristics, and then the AI ​​assistant API and AI generative model fill in the parameters into a standard workflow for creating waveforms.

[0014] Figure 1 An embodiment of a system configuration for an AI assistant used in a testing system is shown. User 20 interacts with generative model 12 via AI-assisted application programming interface (API) 10. Interaction with API 10 can occur directly, for example, by user 20 providing a script to call API 10, or through a user interface such as a chatbot interface or other user input windows that provide prompts. AI assistant 10 can access general test and measurement (T&M) knowledge 14. Based on user configuration, assistant 10 can also access information about waveform impairments 16, such as jitter, noise, etc., and instrument descriptions 18, which will be discussed in more detail below. In addition, assistant 10 can access various protocol standards 26, such as PCIGeNX (Peripheral Component Interface), USBGeNX (Universal Serial Bus), and LAN (Local Area Network), where GenX represents any generation. Furthermore, the system may include one or more test and measurement instruments, such as test and measurement instrument 28. Test and measurement instruments may include oscilloscopes, including mixed-signal oscilloscopes, digital multimeters, spectrum analyzers, etc. Test and measurement instruments may include waveform generation capabilities, such as arbitrary waveform generators. Test and measurement instruments include any instrument capable of receiving signals from the device under test and generating waveforms.

[0015] Generative AI models, including any LLM, can include computer code running on one or more processors (such as processor 24). The processor can reside on one or more computing devices, including test and measurement instruments 28, servers remotely operated from the user, the user's local computing device, etc. The system can include any combination of the aforementioned devices and other computing devices. The resulting waveforms can be stored in storage device 22.

[0016] Through embodiments of the system disclosed herein, user 20 can request a set of waveforms in many different ways. Examples may include, but are not limited to, “Creating 10,000 PCIeGen7 waveforms for all legal symbol rates and amplitudes using SSPRQ (Short Stress Mode Random Quadratic) mode. Randomly varying the Time Interval Error (TIE) for each waveform using a normal distribution with a standard deviation (stddev) of 1 / 4 unit interval (UI). Noise varies randomly between 1% and 10%. Adding metadata to the waveforms using the parameters used to define it.” Another example could be, “Creating 35,000 waveforms based on the symbols, data rates, and amplitudes exported from the waveform file pcie_gen7_03_30.wfm.” Randomly varying the TIE using a normal distribution, where stddev = 0.1UI. Changing the impairment by randomly selecting an s4p file from the directory “cable_models” and using it as the waveform model. Adding metadata to the waveforms using the parameters used to define it. Another example includes "creating 60,000 waveforms by randomly selecting from the 'PCIe7_Examples' directory and using data rate, bit sequence, and amplitude. Then, changing the TIE so that modes 001 and 110 have [...] transition times, and modes 101 and 010 have [...] transition times."

[0017] The outputs of examples of this disclosure include, but are not limited to, analog time-domain waveforms, fast-frame analog data (where “fast-frame” refers to a segmented memory acquisition mode designed to save memory when acquiring a large number of low duty cycle events), waveform databases from fold-created datasets, digital time-domain waveforms, and IQ (in-phase and quadrature waveforms of radio frequency (RF)).

[0018] User 20 can define waveforms using many different parameters. The discussion below provides examples, but is not limited to these. User 20 can define test patterns, which are patterns or sets of symbols that represent data in a pattern and are repeated one or more times, such as pseudo-random binary sequences (PRBS7, 9, 15, 23…), SSPRQ, etc. User 20 can also define modulation techniques, such as Pulse Amplitude Modulation 4 (PAM4), Non-Return-to-Zero (NRZ), and Quadrature Amplitude Modulation (QAM4, QAM16). Some test patterns may imply corresponding modulations; for example, SSPRQ implies PAM4 modulation.

[0019] Some parameters may relate to the data rate of bits or symbols in the mode, the frequency per second of the RF signal carrier frequency, signal amplitude, signal offset, and whether the signal is differential or single-ended. For differential signals, users can also include timing between differential signals. These parameters may also relate to signal impairments, such as inter-channel skew, which indicates timing errors, noise, jitter (including random and deterministic jitter), inter-symbol interference (ISI), and bit and symbol error rates in differential signals.

[0020] Other parameters may include whether to use a spread spectrum clock (SSC), the file for reading S-parameters, waveforms, etc., and specifying the file to write the waveform to or other files. Parameters may include a count of how many waveforms to generate and a statistical specification for randomly specifying the event occurrence rate. Furthermore, user 20 can specify a target instrument, such as an oscilloscope or an arbitrary waveform generator (AWG). User 20 can also provide the signal power, the slope of the signal power over the duration of the RF signal, and the RF pulse repetition interval. Figure 1 As shown, the system can also consider standards such as PCI (Peripheral Component Interface) or PCI-E (PCIexpress), Universal Serial Bus (USB), including protocol-related packets and / or fields.

[0021] User 20 can provide any of the above-mentioned inputs to AI assistant 10. In some embodiments, user input may include: an enumeration, such as PRBS7, PRBS10, etc., for pattern definition; an actual value plus an optional unit, such as data rate, which may be 2.7 GHz, where the transmitted value is 2.7e. 9 The units are Hertz; statistical descriptions, such as a mean of 0.1 UI and a standard deviation of 0.05 UI; a count of the number of requested items, randomly selected, which may include options randomly selected from a set of options, such as files in a directory; and dependencies, such as a definition of the relationship between parameters. The statistical description involves randomly selecting values ​​based on statistical specifications. If a sufficiently large population is measured from the definition, the auxiliary API 10 and AI model approximate the specified statistical definition. For example, a request for 100 waveforms with a data rate of 2.7 GHz mean and 160 MHz standard deviation can generate 100 waveforms, each with a different data rate, such that the data rates in the 100 waveforms have a normal distribution centered at 2.7 GHz.

[0022] In some embodiments, templates and creation rules can be used to define more complex behaviors. For example, the example prompt "100 I2C packets from 3 10-bit devices, where 1 device does not respond" might use a template for 10-bit packets and creation rules specifying how to convert the packets into analog signals "clk" and "data". Templates and creation rules allow complex behaviors to be added without requiring a large amount of code. Template definitions in XML, JSON, YAML, or other descriptive languages ​​allow data templates and creation rules to be defined in a generic way.

[0023] Figure 2 It shows the use of, such as Figure 1 The flowchart shown is an example of a workflow diagram for the system generating waveforms. Figure 2 In this embodiment, the method begins by defining a pattern at point 30. The user can define the pattern in any of the ways described above. As an example, the definition may include data rate, bit error, and / or any other parameters mentioned above. The generative AI model can call a function that provides parameters related to the output waveform. If the user does not describe the parameters, the system can provide reasonable default values. The system provides a timing definition at point 32, and the assistant then sends the pattern and definition to the model at point 34. The model returns the waveform at point 36.

[0024] At point 38, the user can verify the set of waveforms to ensure that it conforms to the specifications given at the process input. For example, if a TIE with a specific standard deviation is set for the process input, the entire set of waveforms can be evaluated to determine if it conforms to the specification. After verification, the system can then store the waveforms for later use, or at point 39, use them in test and measurement instruments.

[0025] After the user requests a set of waveforms, the assistant API and AI model collect information from the user input and use it... Figure 2 The workflow within the framework is used to generate waveforms. In some examples, the helper API and the generative AI model use [a specific method / mechanism] for each requested waveform. Figure 2 The workflow within the system. For example, if a user requests 100 waveforms, the assistant API and the AI ​​model use... Figure 2 The workflow is executed 100 times to generate 100 waveforms. Figure 2 In the exemplary workflow shown, the system-called function provides parameters related to the output waveform. As mentioned above, reasonable default values ​​will be used when the user does not describe the project. Furthermore, the generative AI model can adjust the waveform to be generated to take into account specific characteristics, capabilities, and limitations, such as sampling rate, storage depth, or period-related performance.

[0026] When the system uses waveforms as input, the workflow is as follows: Figure 3 Make the following changes as shown. Figure 3 In the process, the input waveform at position 40 is sent to the waveform analyzer. Then, as... Figure 2 As shown, the process continues as follows: defining the pattern at 42, determining the timing definition at 44, sending information to the model at 46, and receiving the waveform via an assistant at 48. The waveform is verified at 50 to ensure it meets the required specifications, and then used or stored at 52.

[0027] The waveform analyzer uses analysis functions to determine waveform elements such as mode, data rate, and amplitude. If a connection is mentioned in the prompt, these elements are passed to the AI ​​assistant. For example, a user can provide measured characteristics of an existing waveform as partial input, and the AI ​​model can provide a set of waveforms as output, which possess those measured characteristics, with any missing parameters filled in by the assistant.

[0028] In addition to creating new waveform groups, AI models can also enhance existing waveform groups. For example, suppose a particular set of waveforms or a waveform library has a current TIE of 5 picoseconds (ps), but the user wants the entire waveform group to have a peak-to-peak value of 5.6 picoseconds and a standard deviation of 3.2 picoseconds. The AI ​​model can take the current TIE measurement for that group and the desired final TIE measurement and generate additional waveforms that will alter the TIE measurement for that group to meet the desired final TIE measurement.

[0029] While waveform creation is a common task, using AI models combined with domain-specific knowledge can create a large number of unique waveforms with machine learning labels. These waveforms can be indistinguishable from those captured using an oscilloscope and a device under test (DUT). These waveforms are useful in machine learning scenarios or other situations where waveforms cannot be accessed from the DUT, such as in the early days of bus standard creation.

[0030] This disclosure relates to providing protocol standard knowledge and instrument knowledge to be loaded into an AI assistant, allowing users to reference the standards and key aspects of a signal without having to specifically program this knowledge as part of a waveform creation workflow. Furthermore, this disclosure provides a standard workflow for creating output data, where the AI ​​model provides specific values ​​for one or more parameters that the workflow can control. This disclosure relates to workflows where parameter definitions are statistical definitions rather than specific values, meaning that each time the workflow is run, a new, unique waveform is created based on the provided statistical definitions. This disclosure also relates to creating, naming, and storing numerous waveforms based on parameters defined in the prompts.

[0031] The aspects of this disclosure can operate on specially created hardware, firmware, digital signal processors, or specially programmed general-purpose computers including processors that operate according to programmed instructions. The terms controller or processor used herein are intended to include microprocessors, microcomputers, application-specific integrated circuits (ASICs), and special-purpose hardware controllers. One or more aspects of this disclosure can be embodied in computer-usable data and computer-executable instructions, for example, in one or more program modules executed by one or more computers (including monitoring modules) or other devices. Typically, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types when executed by a processor in a computer or other device. Computer-executable instructions can be stored on non-transitory computer-readable media, such as hard disks, optical disks, removable storage media, solid-state storage, random access memory (RAM), etc. As those skilled in the art will understand, the functionality of program modules can be combined or distributed in various aspects as needed. Furthermore, this functionality can be wholly or partially embodied in firmware or hardware equivalents, such as integrated circuits, FPGAs, etc. Specific data structures can be used to more efficiently implement one or more aspects of this disclosure, and these data structures are contemplated within the scope of the computer-executable instructions and computer-usable data described herein.

[0032] In some cases, the disclosed aspects may be implemented in hardware, firmware, software, or any combination thereof. The disclosed aspects may also be implemented as instructions carried on or stored on one or more non-transitory computer-readable media, which may be read and executed by one or more processors. Such instructions may be referred to as a computer program product. As described herein, a computer-readable medium means any medium accessible by a computing device. By way of example and not limitation, a computer-readable medium may include computer storage media and communication media.

[0033] Computer storage media refers to any medium that can be used to store computer-readable information. By way of example and not limitation, computer storage media may include RAM, ROM, electrically erasable programmable read-only memory (EEPROM), flash memory or other storage technologies, optical disc read-only memory (CD-ROM), digital video disc (DVD) or other optical disc storage, magnetic tape cassettes, magnetic tape, disk storage or other magnetic storage devices, and any other volatile or non-volatile, removable or non-removable medium implemented in any technology. Computer storage media does not include the signal itself or the transient form of signal transmission.

[0034] A communication medium is any medium that can be used for computer-readable information communication. By way of example and not limitation, a communication medium may include coaxial cable, fiber optic cable, air, or any other medium suitable for communication of electrical, optical, radio frequency (RF), infrared, acoustic, or other types of signals.

[0035] Example

[0036] Illustrative examples of the disclosed techniques are provided below. An embodiment of these techniques may include one or more examples described below, as well as any combination of these examples.

[0037] Example 1 is a computing device comprising: one or more memories, including test and measurement knowledge; a generative artificial intelligence (AI) model capable of accessing the one or more memories; a display; a user control device allowing a user to provide input; and one or more processors configured to execute code to cause the one or more processors to: access an application programming interface (API) for an AI assistant for the generative AI model to allow a user to interact with the AI ​​assistant; receive one or more user inputs via one or more of the API or user interface, the user inputs providing a description of one or more waveforms to be generated; customize each waveform definition from the description using the AI ​​assistant and access the generative AI model; receive one or more waveforms from the AI ​​assistant; and store one or more waveforms.

[0038] Example 2 is a computing device similar to that of Example 1, wherein one or more processors are further configured to perform Retrieval Enhancement Generation (RAG) to access information outside of memory.

[0039] Example 3 is a computing device of either Example 1 or 2, wherein one or more memories store a corpus of the test and measurement knowledge.

[0040] Example 4: A computing device of any one of Examples 1 to 3, wherein user input includes enumeration, real values, statistical descriptions, counts, and one or more of a set of options.

[0041] Example 5 is a computing device of any of Examples 1 through 4, wherein the code that enables one or more processors to receive one or more user inputs includes code that enables one or more processors to present a user interface with a template having associated creation rules.

[0042] Example 6 is a computing device of any of Examples 1 through 5, wherein the code that enables one or more processors to receive one or more user inputs includes code that enables one or more processors to provide templates and definitions for the inputs.

[0043] Example 7 is a computing device of any of Examples 1 through 6, wherein the code that enables one or more processors to customize the waveform definition includes code that enables one or more processors to use default values ​​when no user input is received.

[0044] Example 8 is a computing device of any of Examples 1 through 7, wherein the code that enables one or more processors to customize a waveform definition includes taking the measured characteristics of an existing waveform as input to generate a new waveform with specified characteristics.

[0045] Example 9 is a computing device similar to Examples 1 through 8, wherein one or more processors are further configured to verify one or more waveforms received from an AI assistant based on waveform definitions.

[0046] Example 10 is a computer-implemented method for generating one or more waveforms, comprising: accessing an application programming interface (API) of an AI assistant for a generative artificial intelligence (AI) model to allow a user to interact with the AI ​​assistant; receiving one or more user inputs via one or more of the API or user interface, the user inputs providing a description of one or more waveforms to be generated; using the AI ​​assistant to customize the waveform definition according to the description, and accessing the generative AI model to receive one or more waveforms generated by the generative AI model from the AI ​​assistant; and storing or using the one or more waveforms.

[0047] Example 11 is a computer implementation of the method in Example 10, which also includes using Retrieval Augmentation (RAG) to access information outside the system where the generative AI model resides.

[0048] Example 12 is a computer-implemented method of either Example 10 or 11, wherein one or more memories store test and measurement knowledge bodies that can be accessed by a generative AI model.

[0049] Example 13 is a computer implementation of any of Examples 10 through 12, wherein user input includes one or more of enumerations, real values, statistical descriptions, counts, and a set of options.

[0050] Example 14 is a computer-implemented method of Examples 10 through 13, wherein presenting a user interface includes presenting a user interface with a template having associated creation rules.

[0051] Example 15 is a computer implementation of the methods of Examples 10 through 14, wherein receiving user input through a user interface includes providing templates and definitions for the input.

[0052] Example 16 is a computer implementation of the method in Examples 10 through 15, where the custom waveform definition includes using a default value when no user input is received.

[0053] Example 17 is a computer-implemented method of Examples 10 through 16, wherein code that enables one or more processors to customize waveform definitions includes taking measurement features of an existing waveform as input to generate a new waveform with the measurement features.

[0054] Example 18 is a computing device similar to Examples 10 through 17, and also includes a waveform definition for verifying one or more waveforms received from an AI assistant.

[0055] The previously described versions of the disclosed subject matter have many advantages that are described or understood by those skilled in the art. However, these advantages or features are not necessarily required in all versions of the disclosed apparatus, system, or method.

[0056] All features disclosed in the specification, including the claims, abstract, and drawings, as well as all steps in any disclosed method or process, may be combined in any combination, except that at least some of such features and / or steps are mutually exclusive combinations. Unless otherwise expressly stated, each feature disclosed in the specification, including the claims, abstract, and drawings, may be replaced by an alternative feature for the same, equivalent, or similar purpose.

[0057] Furthermore, specific features are mentioned in this written description. It should be understood that the disclosure in this specification includes all possible combinations of these specific features. For example, where a specific feature is disclosed in the context of a particular aspect, that feature may also be used in the context of other aspects to the extent possible.

[0058] Furthermore, when a method having two or more defined steps or operations is mentioned in this application, the defined steps or operations may be performed in any order or simultaneously, unless the context precludes such possibilities.

[0059] While specific aspects of this disclosure have been shown and described for illustrative purposes, it should be understood that various modifications may be made without departing from the spirit and scope of the invention. Therefore, the invention should not be limited to anything other than the appended claims.

Claims

1. A computing device, comprising: One or more memories, including test and measurement knowledge; Generative artificial intelligence (AI) models that are able to access one or more of the aforementioned memories; monitor; User control devices that allow users to provide input; and One or more processors are configured to execute code that causes the one or more processors to: Access the application programming interface (API) of the AI ​​assistant used for generative AI models to allow users to interact with the AI ​​assistant; Receive one or more user inputs through one or more of the APIs or user interfaces, the user inputs providing a description of one or more waveforms to be generated; Use the AI ​​assistant to customize each waveform definition from the description and access the generative AI model; Receive one or more waveforms from the AI ​​assistant; and Store the one or more waveforms.

2. The computing device of claim 1, wherein one or more processors are further configured to perform Retrieval Enhancement Generation (RAG) to access information outside of memory.

3. The computing device of claim 1, wherein the one or more memories store a corpus of the test and measurement knowledge.

4. The computing device of claim 1, wherein the user input includes one or more of enumerations, real values, statistical descriptions, counts, and selections from a set of options.

5. The computing device of claim 1, wherein the code that causes the one or more processors to receive the one or more user inputs includes code that causes the one or more processors to present a user interface having a template with associated creation rules.

6. The computing device of claim 1, wherein the code that causes the one or more processors to receive the one or more user inputs includes code that causes the one or more processors to provide templates and definitions for the inputs.

7. The computing device of claim 1, wherein the code that causes the one or more processors to customize the waveform definition includes code that causes the at least one processor to use a default value when no user input is received.

8. The computing device of claim 1, wherein the code that causes the one or more processors to customize the waveform definition includes taking measurement features of an existing waveform as input to generate a new waveform with specified features.

9. The computing device of claim 1, wherein the one or more processors are further configured to verify one or more waveforms received from the AI ​​assistant according to the waveform definition.

10. A computer-implemented method for generating one or more waveforms, comprising: Access the application programming interface (API) of the AI ​​assistant used for generative artificial intelligence (AI) models to allow users to interact with the AI ​​assistant; Receive one or more user inputs through one or more of the APIs or user interfaces, the user inputs providing a description of one or more waveforms to be generated; Use the AI ​​assistant to customize waveform definitions from descriptions and access generative AI models; Receive one or more waveforms generated by a generative AI model from the AI ​​assistant; and Store or use the one or more waveforms.

11. The computer-implemented method of claim 10, further comprising using Retrieval Augmentation (RAG) to access information outside the system where the generative AI model resides.

12. The computer-implemented method of claim 10, wherein the one or more memories store test and measurement knowledge bodies accessible by the generative AI model.

13. The computer-implemented method of claim 10, wherein the user input includes one or more of enumeration, real values, statistical descriptions, counts, and selections from a set of options.

14. The computer-implemented method of claim 10, wherein presenting the user interface includes presenting a user interface having a template with associated creation rules.

15. The computer-implemented method of claim 10, wherein receiving user input via a user interface includes providing templates and definitions for the input.

16. The computer-implemented method of claim 10, wherein the custom waveform definition includes using a default value when no user input is received.

17. The computer-implemented method of claim 10, wherein the code that causes the one or more processors to customize the waveform definition includes taking measurement features of an existing waveform as input to generate a new waveform having the measurement features.

18. The computer-implemented method of claim 10, further comprising verifying one or more waveforms received from the AI ​​assistant against a waveform definition.

Images