AI embedded vector database calibration architecture

Abstract

A test and measurement system has a test and measurement instrument including: a connection to a device under test (DUT); one or more analog-to-digital converters (ADCs) for receiving the signal from the DUT and converting it into one or more digital waveforms; and one or more processors to: receive one or more digital waveforms corresponding to one set of adjustment parameters applied to the DUT; constructing one or more image tensors of the one or more digital waveforms; using an artificial intelligence embedding model, generating one or more text character strings according to the metadata, and embedding the metadata and the one or more image tensors into a vector; accessing a vector database; receiving a set of indices having a number of indices corresponding to the number of matches; using the index set to find one or more optimal adjustment parameter sets; and verifying the operation of the DUT.

Description

[0001] Cross-references to related applications

[0002] This disclosure is a claim on U.S. Provisional Application No. 63 / 642,629, filed May 3, 2024, entitled “AI Embedded Vector Database Calibration Architecture,” and U.S. Provisional Application No. 63 / 717,842, filed November 7, 2024, entitled “AI Embedded Vector Database Architecture for Calibrating Test Devices in Manufacturing,” the disclosures of which are incorporated herein by reference in their entirety. Technical Field

[0003] This disclosure relates to test and measurement systems and methods, and more specifically, to applying machine learning to the measurement of signals from a device under test (DUT). Background Technology

[0004] U.S. Patent No. 11,923,896, entitled "Optical Transmitter Tuning Using Machine Learning," published March 5, 2024, hereinafter referred to as the "'896 Patent," is incorporated herein by reference. Similarly, U.S. Patent No. 11,923,895, entitled "Optical Transmitter Tuning Using Machine Learning and Reference Parameters," also published March 5, 2024, hereinafter referred to as the "'895 Patent," is incorporated herein by reference. Both describe systems and methods for applying trained machine learning systems to predict optimal tuning parameters for optical transceiver devices (DUTs). Embodiments of these systems may also be referred to herein as OptaML and / or OptaML Pro.

[0005] Other Tektronix patents and patent applications describe other ways to improve the resolution of key features, extend the recording length of segments placed in an image, and use the image space more efficiently to extract even more feature sets. For example, U.S. Patent Application No. 17 / 747,954, filed May 18, 2022, entitled “Machine Learning for Measurement Based on a Short-Mode Waveform Database,” hereinafter referred to as the “'954 application,” the contents of which are incorporated herein by reference, describes a system and method for applying a trained machine learning system to make predictions about measurements of signals received from a device under test (DUT). When the DUT is, for example, an optical transceiver, an example signal measurement is a transmitter and a dispersive eye closed quaternion (TDECQ). The machine learning system described in the '954 application is used as input to both the training and runtime of the machine learning system in the form of tensor images constructed using graphical representations of specific short-mode segments within long waveforms acquired from the DUT.

[0006] As another example, U.S. Patent Application No. 18 / 199,846, filed May 19, 2023, entitled “Automated Cavity Filter Tuning Using Machine Learning,” hereinafter the “‘846 Application,” the contents of which are hereby incorporated by reference into this application, describes systems and methods for applying a trained machine learning system to make predictions of the optimal position of a tuning element in a tunable device under test, such as a cavity filter. The input to the machine learning system described in the ‘846 Application, both for training and runtime, is in the form of tensor images, constructed using a graphical representation of scattering parameters (S-parameters) or other parameters characterizing the DUT.

[0007] As another example, U.S. Patent Application No. 18 / 665,258, filed November 28, 2024, entitled “User Interface for Tensor Builder to Build Images for Input to Machine Learning,” hereinafter the “‘258 Application,” the contents of which are hereby incorporated by reference into this application in their entirety, describes systems and methods that allow a user to select a type of tensor and construct the selected type of tensor from data from a DUT. The tensor is then sent to a machine learning network.

[0008] In previous systems, there was no way to efficiently update the training of a neural network with new data from a device that could not be predicted, but could still be tuned through a user process. Updating these previous systems required a time-consuming and tedious complex retraining of the neural network. In addition, a drawback of previous systems was that they were trained with a given number of tuning parameters for only one model of DUT or one type of DUT. If it was desired to use a different model of DUT with a different number of tuning parameters, then the system had to be retrained.

[0009] There is a need for a system that can work on all DUTs with any number of tuning parameters. A user should have the ability to update these systems to tune new types of DUTs and add new DUT tuning data without having to train or retrain a neural network. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 A waveform image tensor is shown.

[0011] Figure 2A And 2B Block diagrams of embodiments of a machine learning system during training and runtime, respectively, are shown.

[0012] Figure 3 A block diagram of an embodiment of an AI vector embedding system is shown.

[0013] Figure 4A And 4BEmbodiments of an AI vector embedding system are shown that illustrate vector database interactions and system behavior. DETAILED DESCRIPTION

[0014] Embodiments herein relate to artificial intelligence (AI) to embed waveform image tensors as vectors into a vector database system. This eliminates the need to train a neural network. These embodiments relate to a combination of pre-trained neural networks within a vector database, and a visual transformer and a text transformer. These embodiments rely on tensor construction and feature extraction to support a specific model of a device under test (DUT). These capabilities exist within Tektronix’s OptaML TM System, so users of that system can integrate these embodiments into those systems. Further, embodiments include a new regression classifier that can also be integrated into existing systems. With the embodiments disclosed here, new users can get these capabilities.

[0015] Embodiments employ a vector database system that allows users to store and search vectors of complex information. It should be noted that this particular implementation is not limited to using any particular vector database system, nor is it limited to any particular module for relating images to their text descriptions, which is typically external to the vector database system, with information being passed between the module and the database.

[0016] However, for ease of understanding, the embodiments here employ a subscription-based vector database called Pinecone. Further, the embodiments employ CLIP. While Pinecone provides an application programming interface (API) for several image-text pair embedding models, including CLIP from OpenAI. CLIP (Contrastive Language-Image Preprocessing) can relate images to their text descriptors using a combination of image recognition and natural language processing. Other terms in this discussion have specific meanings. As used here, “LLM” refers to a large language model. “ML” refers to machine learning. “MFG” refers to manufacturing. “RAG” refers to regenerative augmentation generation, a means of providing additional information to an LLM. The term “tensor” refers to an image containing extracted waveform features and metadata. It represents characteristics of a DUT and how its tuning parameters and metadata affect waveforms.

[0017] The primary goal of the system described in the ‘895 and ‘896 patents relates to tuning DUT parameters on a manufacturing line. In a one-shot tuning prediction, the process feeds a tensor image from a current DUT into a neural network and gets a set of tuning parameter numbers to store directly into registers of the DUT. Users no longer need to sweep all tuning parameters until the tuning process finds the best set of tuning parameters. This results in a significant acceleration of tuning time on a manufacturing line.

[0018] Figure 1 An embodiment of a waveform tensor image 10 is shown. The image contains segments of three waveforms 12, each representing a different one of three sets of tuning parameters. The image can also include bar graphs (such as 14 and 16) representing additional information such as temperature and noise, as well as short pattern segments (such as 18) from the waveforms, with multiple instances overlaid.

[0019] Figure 2A and 2B is a block diagram illustrating training and runtime operations of an example of a prior machine learning system. In Figure 2A the neural network 26, in one embodiment a Resnetl8 neural network, undergoes training to receive image tensors containing waveform segments and other metadata such as temperature and noise. In one example, at 20, a user collects waveforms and data from a number of devices under test (DUTs). For example, the user can collect waveforms and data for 200 DUTs, making an array of 200 image tensors at 22. The user also tunes each DUT and makes an array of 200 best tunings to be consistent with the image tensors for each DUT at 24. These two arrays are presented to the neural network 26 and the training operation adapts the neural network, training the neural network to associate image tensors with sets of best tunings. In one embodiment, the outer three layers of Resnetl8 are trained to associate image tensors with sets of best tunings. The remaining layers in the pre-trained Resnetl8 network remain unchanged in the transfer learning process.

[0020] Figure 2B Operation of the trained system during runtime is shown, where the neural network has been trained on a particular model of DUT with a particular number of tuning parameters. During runtime of these prior systems (neural network 26), it can be used to make tuning predictions for each DUT during the manufacturing process. At 28, a user loads a DUT to be tuned with the same three reference parameter sets that were used during training and collects data about the DUT. The tensor builder 22 then builds a tensor image using the three waveforms acquired from the DUT and any other information that can be included in bar graphs, etc. The neural network 26 receives the resulting tensor. The neural network outputs a set of best tuning parameters, which are register values to be stored in registers of the DUT. The user can then verify the tuning. If the DUT passes, the user moves on to the next DUT. If it fails, the user will attempt to tune with their original slower algorithm to make a final decision about pass or fail.

[0021] The embodiments herein generally use the advantage of embedding AI vectors into a database to obtain the advantage of not having to train a neural network to add new DUT models or new data for existing models. The AI embedding model can require pre-training, but if so, it is done. If the text plus image embedding vectors are sufficiently different and unique for all examples, then no pre-training is needed. The set of pre-training would depend on the ability of the embedding model to adequately represent tensor differences that are important to associate with the adjustment parameters.

[0022] The embodiments herein generally use similar DUT characterization techniques that use three or more sets of reference parameters, for example, as described in the ‘895 patent or the ‘846 application. Like the ‘895 patent, embodiments of the present disclosure also use a similar tensor constructor to place three or more captured waveforms into an image tensor. However, some embodiments of the present disclosure can also use text language embeddings associated with the DUT to obtain enhanced capabilities.

[0023] The embodiments herein differ from previous systems by using AI embedding models and a vector database. For example, they generally still use a pre-trained neural network, such as Resnetl8. However, the neural network does not undergo training for direct output of adjustment parameters as done in previous systems such as Figure 2A and 2B Instead, the neural network undergoes training to output a vector in a multi-dimensional space. The image tensor is converted into a vector embedding that is stored and indexed into a vector database. This mechanism turns the deep learning network into a component that can be used for all new DUT training data or new DUT model types.

[0024] As mentioned above, the embodiments herein employ a CLIP embedding model. Other types of embedding models, such as BLIP / 2, FLIP, SigLIP, etc., perform similar functions, performing similar types of image-text pairings. These models take an image and a text language string as input and convert them into a vector to be stored in a vector database. There are various types of AI embedding models available, each specialized for different data types, such as language, audio, images, and video. Some embodiments of the present disclosure use the CLIP model, which incorporates Resnetl8, a pre-trained deep learning network that performs well in low-level feature extraction of waveform tensors. However, embodiments are not limited to using the CLIP model. Networks like the CLIP model can extract key waveform features that are most associated with adjustment parameters. The CLIP model also contains a ViT (Visual Transformer), which can provide additional advantages. Furthermore, the model includes a language transformer that can embed language associated with the image.

[0025] Additional text can be used to aid vector matching by including metadata such as temperature of the DUT, measured noise on the waveforms that are filtered out for feature extraction, DUT model, and any other relevant information. This allows the system to isolate the vector database match search into smaller sets of vectors, speeding up the search process and reducing the cost of the subscription service or other vector database applications that can be used in embodiments of the present disclosure.

[0026] The AI embedding model can require pre-training to better handle the variety of expected input tensor images and text. This pre-training further enables the goal that this AI embedding model, once deployed, will work with a large number of DUTs that all generate similar varieties of tensor images. The metadata text with the measurements, model, and other metadata makes this feasible because these are important parts of reducing the set of vectors to search to make an adjustment match. This mechanism takes some of the load of image resolution differentiation off of the tensor image and Resnetl8 neural network. This allows the neural network to work over a wider range of DUT types and conditions, making the system more generalized and able to handle a wider range of use cases without retraining.

[0027] Figure 3 A block diagram of the test and measurement system is shown for both training and runtime. The left side of the diagram shows the system in pre-training 30. The right side shows the system in runtime 30. While the system components are the same, and shown twice for ease of understanding, the pre-training and runtime processes are connected when the runtime system does not provide the parameters needed by the DUT through testing. If these system components result in or are produced by different processes, they can have different reference numbers between the two processes.

[0028] An external computing device 49 is shown bridging between the two. The computing device can include a manufacturing computer that operates the user testing process and interacts with the test and measurement instrument 72, vector database 44, and AI embedding model 40 from Figure 2B The AI embedding model can include code executed by one or more processors on the external computing device, or can include code operating on a remote server. The vector database can also operate as a local database on the computing device, or can also run on a separate server. The test and measurement system has one or more processors positioned anywhere in the system that execute code to carry out various operations and tasks.

[0029] The system’s pre-training involves collecting data from many DUTs as a primary starting point. In some embodiments, data is collected from approximately 200 nominal DUTs at 32. This process has many similarities to the prior art OptaML system. Figure 3 The system of the present disclosure also determines a certain number of the set of reference parameters to load into all DUTs, the number of waveforms to collect from each set of each DUT, extracts features from the waveforms, and places them into the tensor image 34 and language text 38. This process produces an array of tensor images plus text that represents the variation of the group of DUTs. This process also includes best tuning these sample DUTs at 36 and saving the best tuning set into an array of best tuning sets 46.

[0030] Unlike the previous system, the system of the embodiments does not use the array of best tuning parameters and the array of tensor images to train a neural network. Instead, the array of tuning is saved for later retrieval of the best tuning set. The array of tensor images is input into the CLIP AI embedding model to convert it into an array of indices of vectors. These vectors are stored in a vector database, and the indices associated with these vectors are the same indices in the associated array of tuning parameters 46. Once the vector database contains a set of vectors representing many different DUTs, and the associated array of best tuning parameters is created, the system is ready for use on the MFG in a runtime state to tune new DUTs.

[0031] Once the system has been trained, meaning the vector database has been populated with a set of embedding vectors from the AI embedding model, a user can begin using the system during runtime 60. The user then takes up a single device, understands that some manufacturing lines can operate on multiple DUTs in parallel, and gathers the resulting waveforms from the DUTs at 62. The tensor builder 34 builds the tensor, and the embedding model 40 and neural network 42 operate as before to provide text from the text transformer 38 and visual transformer 48 if used, to generate the embedded AI vectors. The vector database 44 then performs the vector matching process 64 to produce a set of vectors that contain the best tuning parameters for the DUT at 66. The tuning parameters are extracted from the vectors and placed in the DUT at 70. The process then tests the DUT at 72. If the DUT passes at 74, the system returns to the runtime side and continues to move to test the next DUT(s).

[0032] If the DUT fails with one of the sets of adjustment parameters, the process can iterate. Unlike the previous system, the vector matching process produces a certain number of vectors and thus a certain number of sets of adjustment parameters from the best adjustment parameter array 66 closely match the best adjustment parameters at runtime. The iteration involves placing other adjustment parameters from the adjustment parameter number on the DUT and then testing. If at 74, all of the sets of adjustment parameters fail, the process moves on to adjust the DUT at 76. At 76, the DUT is adjusted by the user’s previous process. If the DUT still fails at this point, it is discarded or sent out for repair. If the DUT is adjusted, the image tensor built for this DUT is retrieved from the tensor builder at runtime and then this information is used to generate a new embedding vector with the image tensor from the runtime side, updated with the successive adjustment parameters and related text. This is then used to update the vector database. As new DUT models are added and performance is optimized, this ensures that the database continues to grow and improve over time. The iterative process of adjusting and adding new models to the database helps improve the accuracy and efficiency of the system, ultimately leading to better results and higher productivity.

[0033] However, if the DUT is successfully adjusted, the new tensor image and the best set of adjustment parameters are added to the vector database and the best adjustment array, respectively. As new DUT models are added and performance is optimized, this ensures that the database continues to grow and improve over time. The iterative process of adjusting and adding new models to the database helps improve the accuracy and efficiency of the system, ultimately leading to better results and higher productivity.

[0034] The vector database 50 can include a Pinecone database, a subscription service for vector databases. This particular vector database provides a wide range of functionality to manage and grow the database while in use. Currently, it is a good choice to implement code with minimal effort because it provides high-level commands that make it very easy to use. Pinecone also supports many different types of AI embedding models that can be specified for creating vector embeddings, such as CLIP. However, embodiments of the present disclosure are not limited to using Pinecone.

[0035] In addition to supporting various AI embedding models, Pinecone also provides multiple types of queries and retrieving data from the vector database. The embodiments herein generally use image-to-image queries to convert images into vectors, which use the CLIP AI embedding model. These are vectors that are used to access the best-tuned parameter array and read out the best-tuned parameters. This process ensures that new DUTs are best-tuned because the system identifies the best-tuned parameters by comparing the vectorized images of the new DUTs to those of the DUT models stored in the database.

[0036] Even if the closest matching adjustment is not the best, it should be very close to the best, which means that the user algorithm for scan tuning can start from a close starting point. As a result, the scan tuning process should take less time compared to starting from a normal default state. This is because the closest matching adjustment provides a good starting point for the user algorithm, reducing the amount of time and effort required to tune the DUT. By using the vector database and the closest matching adjustment as a starting point, the system can significantly improve the efficiency and accuracy of the tuning process, ultimately leading to better results and higher productivity.

[0037] As the vector database grows and more DUT models are added and tuned, it becomes increasingly likely that a one-time tuning will be sufficient for each new DUT. This means that as the database becomes more comprehensive and accurate, the number of times the user optimization algorithm will be required over time will decrease.

[0038] If the AI embedding model does not produce the resolution required for the image matching differences, it can be retrained once to better adapt to more input variations. However, it is expected that only one training will be necessary, and no additional training will be required during runtime as new data is added to the vector database. By retraining the AI embedding model, the system can improve its accuracy and efficiency in identifying the best-tuned parameters for new DUTs. However, it is expected that such retraining will only be necessary in rare cases, and the system will continue to work reliably and accurately with the existing vector database and AI embedding model. Ultimately, the goal is to create a self-sufficient system that requires minimal maintenance or tuning, allowing engineers to focus on other aspects of their work without worrying about the tuning process.

[0039] By using the vector database and the closest matching adjustment as a starting point, Figure 2A and 2B with Figure 3Comparatively, the differences between the prior art adjustment system and the new adjustment system can be seen. The main difference between the two is that in the prior art system, the neural network undergoes training on a fixed set of labels and cannot add new labels without retraining. In contrast, the new system uses a neural network that outputs generalized image embedding vectors that are stored in a vector database. This allows new labels and DUT models to be added without the need to retrain the neural network. The optimal adjustment set is produced by matching input images to existing images in the vector database. This approach allows the system to quickly and accurately identify the optimal adjustment parameters for each new DUT model type added to the system without requiring extensive training or retraining of the neural network. Overall, the new adjustment system is more flexible, efficient, and scalable than the prior art system, making it easier for engineers to optimize the performance of new DUT models and improve overall productivity.

[0040] Figure 4A and 4B An embodiment of the overall system is shown, with more detail shown regarding the interaction between the various parts. The two figures are designed to be placed side by side to show the connections between all the components of the system. The overall system includes a tensor builder 34, an AI embedding model 40, a vector database 50, and a post-processing classifier module 110. As shown at 80, the system operates in an add new data mode or a run-time mode. Thus, the data includes data for a new DUT that did not previously exist in the vector database, or data for a current DUT that is being tested. The tensor builder 34 places three or more captured waveforms into an image tensor. The tensor builder can extract S-parameters 82, short pattern waveforms 84, DSP transforms 86 such as a fast Fourier transform, a bispectrum transform, filtering, correlation, etc., and measurements 88. The measurements can include frequency, amplitude, pulse width, TDECQ (transmitter and chromatic eye closure quaternions), noise, temperature, etc., all of which are previously included in the image tensor. In addition, metadata for the DUT can be gathered, including model number, vendor, type, etc. At 90, the tensor builder creates an RGB image tensor, where different information can be placed in different color channels of the image. In addition, at 91, the tensor builder also makes a text string of the different information. These are passed to the AI embedding model, such as the previously mentioned CLIP. The text string also goes to the vector database, as shown at Figure 4B .

[0041] The AI embedding model 40 takes the RGB image and uses the neural network 42, possibly in conjunction with a visual transformer 48, to develop one part of the combined vector. The text transformer / encoder 38 creates another part, resulting in a combined embedding vector in a multi-dimensional space that is passed to the vector database 50 in Figure 4B .

[0042] exist Figure 4A In the middle, switches 93 and 95 are located at the "Neural Network" position, which allows selection between the Neural Network and the Visual Transformer.

[0043] exist Figure 4B In the vector database, the vector database receives a combined vector from database 50, a text string from creating text string blocks 91, and a vector from... Figure 4A Data collected from the DUT at position 32. At position 94, the combined vectors are written to the database, along with measurement metadata 96, any other classification metadata 98, and tuning parameter metadata 100. This data is also passed to the metadata block at position 104. At position 64, the vector database 50 performs vector matching, aiming to find a set of N closest matches at position 92. This set of closest matches 92 is then passed to the classifier 110. Furthermore, the vector database then filters the metadata at position 102 to allow vector matching to have closer results, thereby speeding up the matching and reducing the amount of time required to search for vectors.

[0044] Then, classifier 110 obtains the vector IDs of the N closest matching sets. At 108, processing algorithm 106 operates to reduce the matching set. This block receives the N closest matching DUT embedding IDs from the vector database. The closest match is relative to the new DUT to be tuned on the MFG line. A waveform or other type of representation data is acquired from the DUT using reference parameters stored in the DUT. Features and measurements are then extracted from the waveform and incorporated into image tensors and text tensors, along with metadata, to obtain a set of embedding objects to be compared with the embeddings in the database. This is done via metadata in the embedding vector ID structure to further partition the vectors into classification groups. Interpolation block 118 reads the tuning parameters from the metadata set stored in the embedding IDs and performs regression analysis on each tuning parameter, calculating the most probable value of the tuning parameter. This is one of the tuning parameter sets that will be returned to the user for trial in their DUT. Other tuning parameter sets will also be returned to the user for trial in their DUT. Users can try these different sets in their DUT to increase their chances of finding the tuning parameter set that meets their requirements for tuning the DUT.

[0045] As discussed above, if the closest match is unsatisfactory, the system can try several next closest matches to obtain better results. Additionally, the user can try the returned set of regression pseudo-interpolation adjustment parameters. If a match is deemed optimal, the system moves on to the next DUT for adjustment. This iterative process ensures that the adjustment parameters chosen for each DUT are validated and optimized to achieve the best possible performance.

[0046] Figure 4A and 4B The switches in both show two states of operation. The first state is new data, and the second state is runtime. The runtime state is for when a new DUT is to be adjusted. Its AI embedding is compared to the embeddings in the database, and the N closest matches of the vector are returned. These IDs are then processed by the regression classifier block to reduce the set and further classify for better accuracy. The closest matches are used to retrieve the best adjustment parameters from the metadata of the IDs to obtain the optical adjustment parameters.

[0047] When new data is selected at block 120, the switches change to the new data setting at each switch. At block 122, the signal is sent to Figure 4B the switch 99 in Figure 4A new data collection from the DUT as from block 32 in . In addition, the switch 97 goes to NC, which disconnects the classifier from the vector database. More data can be added to the vector database without the need to retrain the neural network.

[0048] The resulting system has higher accuracy when making predictions on data that has never been seen before. In addition to images, this architecture also includes text as part of the vector. This relieves a lot of the burden of image interpretation by the neural network. The text metadata is also used to select the folder in the database that the vector goes into. This also results in faster vector matching operations because the search is done on a smaller set of vectors.

[0049] Once the system better“sees” the waveform tensor, the goal is to train (if needed) that will only be done once and work on all future calibration setups that use the tensor builder images. This will be effective if the embedding of each different tensor image / text is unique and different enough to be detected in the cosine matching algorithm. All output predictions are the best set of adjustments because they are derived from the best adjustment array, not from potentially invalid outputs from the neural network.

[0050] Use cases and applications of the embodiments herein include one or more of the following, by way of example only. One use case includes adjusting optical emitters on a manufacturing line, saving a lot of MFG time. Another includes adjusting oscilloscope non-linearity correction and / or frequency response calibration parameters on MFG to save time. These embodiments reduce the dataset scans used to test the die before mounting it into a package to save MFG time. Another includes duplexers and duplexer adjustment for wireless communication antennas. This case uses S-parameters as the tensor image input.

[0051] Aspects of the disclosure can operate on specially created hardware, firmware, digital signal processors, or specially programmed general purpose computers including processors operating according to programmed instructions. The term controller or processor as used herein is intended to encompass microprocessors, microcomputers, application specific integrated circuits (ASICs), and special purpose hardware controllers. One or more aspects of the disclosure can be embodied in computer-usable data and computer-executable instructions, such as embodied in one or more program modules executed by one or more computers (including monitoring modules) or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types when executed by a processor in a computer or other device. Computer-executable instructions can be stored on a non-transitory computer-readable medium such as a hard disk, optical disk, removable storage media, solid state memory, random access memory (RAM), etc. As will be appreciated by one of ordinary skill in the art, the functionality of a program module can be combined or distributed as desired in various aspects. Moreover, functionality can be implemented entirely in firmware or hardware equivalents such as integrated circuits, FPGAs, and the like. Particular data structures can be used to more effectively implement one or more aspects of the disclosure, and such data structures are contemplated as being within the scope of computer- executable instructions and computer-usable data described herein.

[0052] In some cases, the disclosed aspects can be implemented in hardware, firmware, software, or any combination thereof. The disclosed aspects can also be implemented as instructions executed by one or more or non-transitory computer-readable media stored thereon, which can be read and executed by one or more processors. Such instructions can be referred to as a computer program product. Computer-readable media, as discussed herein, means any medium accessible by a computing device. Computer-readable media includes, but is not limited to, computer storage media and communications media.

[0053] Computer storage media means any medium for storing computer-readable information. Examples of computer storage media include, but are not limited to, RAM, ROM, electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact discs read-only memory (CD-ROM), digital video disc (DVD), or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, and any other medium which can be used to store computer-readable information in any technology. Computer storage media excludes transitory signals per se.

[0054] Communication media includes any medium that can be used to communicate

[0055] Examples

[0056] The following provides illustrative examples of the disclosed technology. Embodiments of the technology can include one or a combination of the examples described below and any combination of the examples described below.

[0057] Example 1 is a test and measurement system comprising: a test and measurement instrument comprising: a connection that allows the test and measurement instrument to connect to a device under test (DUT); one or more analog-to-digital converters (ADCs) to receive a signal from the DUT and convert it into one or more digital waveforms; and one or more processors configured to execute code that causes the one or more processors to: receive the one or more digital waveforms from the one or more ADCs, each of the one or more digital waveforms corresponding to a set of tuning parameters applied to the DUT; construct one or more image tensors of the one or more digital waveforms; use an artificial intelligence embedding model that generates one or more text strings from metadata associated with the waveforms and embeds the metadata and the one or more image tensors into a vector; access a vector database; receive a set of indices from the vector database, the set of indices having a number of indices corresponding to a number of matches; use the set of indices to access an array of best tuning parameters to find one or more sets of best tuning parameters for the DUT; and validate operation of the DUT with the one or more sets of best tuning parameters from the array of best tuning parameters.

[0058] Example 2 is the test and measurement system of example 1, wherein the code that causes the one or more processors to validate operation of the DUT comprises code that causes the one or more processors to: tune the DUT with one of the one or more sets of best tuning parameters; test the DUT with one of the one or more sets of best tuning parameters; determine whether the DUT passed the test; use another of the one or more sets of best tuning parameters when the DUT test fails; and repeat until one of: the DUT has passed the test or all of the one or more sets of best tuning parameters have been tested.

[0059] Example 3 is the test and measurement system of example 2, wherein the one or more processors are further configured to execute code that causes the one or more processors to employ a user testing process.

[0060] Example 4 is the test and measurement system of Example 3, wherein the code that causes the one or more processors to employ the user test procedure includes code that causes the one or more processors to use the closest set of the one or more optimal set of tuning parameters to begin the user test procedure.

[0061] Example 5 is the test and measurement system of Example 3, wherein the code that causes the one or more processors to employ the user test procedure includes code that causes the one or more processors to update the artificial intelligence embedding model with text strings from metadata associated with the waveform and the one or more image tensors.

[0062] Example 6 is the test and measurement system of Example 3, wherein the code that causes the one or more processors to employ the user test procedure includes code that causes the one or more processors to discard or repair the DUT when the DUT cannot be tuned.

[0063] Example 7 is the test and measurement system of any of Examples 1-6, wherein the one or more processors are further configured to execute code that causes the one or more processors to use the metadata to reduce the number of indices before the one or more processors use the indices.

[0064] Example 8 is the test and measurement system of any of Examples 1-7, wherein the one or more processors are further configured to execute code that causes the one or more processors to read tuning parameters from the metadata and perform a regression analysis to determine the most likely values of the tuning parameters and return a set of regression tuning parameters.

[0065] Example 9 is the test and measurement system of Example 8, wherein the one or more processors are further configured to use the set of regression tuning parameters when the DUT cannot be tuned from the indices returned by the artificial intelligence vector system or the user test procedure.

[0066] Example 10 is the test and measurement system of any of Examples 1-9, wherein the one or more processors are further configured to execute code to cause the one or more processors to train the artificial intelligence embedding model.

[0067] Example 11 is the test and measurement system of any of Examples 1-10, wherein the code that causes the one or more processors to train the artificial intelligence embedding model includes code that causes the one or more processors to: collect tuning data for a predetermined number of DUTs, the tuning data including optimal tuning parameter arrays, tensor images, and text; and input the tuning data to the artificial intelligence embedding model to allow the artificial intelligence embedding model to convert the tuning data and update a vector database.

[0068] Example 12 is a method of testing a device under test (DUT) comprising: receiving one or more digital waveforms from one or more analog-to-digital converters (ADCs), each digital waveform corresponding to a set of tuning parameters applied to the DUT; constructing one or more image tensors of the one or more digital waveforms; using an artificial intelligence embedding model that generates one or more text strings from metadata associated with the waveforms and embeds the metadata and the one or more image tensors; accessing a vector database to compare the vector to vectors in the database; receiving a set of indices from the vector database, the set of indices having a number of indices corresponding to a number of matchers; using the set of indices to access an array of best tuning parameters to find one or more sets of best tuning parameters for the DUT; and validating operation of the DUT with one of the one or more sets of best tuning parameters from the array of best tuning parameters.

[0069] Example 13 is the method claimed in claim 12, wherein validating operation of the DUT comprises: tuning the DUT with one of the one or more sets of best tuning parameters; testing the DUT with one of the one or more sets of best tuning parameters using a test and measurement instrument; determining whether the DUT passed the test; using another of the one or more sets of best tuning parameters when the DUT test fails; and repeating until one of: the DUT has passed the test, or all of the one or more sets of best tuning parameters have been tested.

[0070] Example 14 is the method of example 13, further comprising employing a user test procedure.

[0071] Example 15 is the method of example 14, wherein employing a user test procedure comprises starting the user test procedure using the closest one of the one or more sets of best tuning parameters.

[0072] Example 16 is the method of example 14, wherein employing a user test procedure comprises updating the vector database with a new vector generated from the one or more text strings from the metadata associated with the waveforms, the one or more image tensors, and the metadata by using the artificial intelligence embedding model.

[0073] Example 17 is the method of example 14, wherein employing a user test procedure comprises discarding or repairing the DUT when the DUT cannot be tuned.

[0074] Example 18 is the method of any of examples 12 to 17, further comprising using the metadata to reduce the number of indices before the one or more processors use the indices.

[0075] Example 19 is the method of any of Examples 12-18, further comprising reading adjustment parameters from the metadata and performing a regression analysis to determine the most likely values of the adjustment parameters and returning a set of regression adjustment parameters.

[0076] Example 20 is the method of any of Examples 12-19, further comprising using the set of regression adjustment parameters when the DUT cannot be adjusted through the indexes returned from the vector database or a user testing process.

[0077] Example 21 is the method of any of Examples 12-20, further comprising training an artificial intelligence embedding model.

[0078] Example 22 is the test and measurement instrument of Example 21, wherein training the artificial intelligence embedding model comprises: collecting adjustment data regarding a predetermined number of DUTs, the adjustment data comprising optimal adjustment parameter arrays, tensor images, and text, the foregoing forming vectors; and inputting the vectors into the vector database to populate the vector database.

[0079] The foregoing described versions of the disclosed subject matter have many advantages over the prior art, some of which have been described and others will be apparent to those of ordinary skill in the art. None of these advantages, however, are necessary essential to all versions of the disclosed devices, systems or methods.

[0080] Moreover, the written description references particular features. It is to be understood that the disclosure in this specification includes all possible combinations for these particular features. Where a particular feature is disclosed in the context of a particular aspect or example, that feature can also be used, to the extent possible, in the context of other aspects and examples.

[0081] Furthermore, when reference is made in this application to a method having two or more defined steps or operations, the defined steps or operations can be carried out in any order or simultaneously, unless the context excludes those possibilities.

Claims

1. A test and measurement system comprising: a test and measurement instrument comprising: a connection that allows the test and measurement instrument to connect to a device under test (DUT); one or more analog-to-digital converters (ADCs) to receive a signal from the DUT and convert the signal into one or more digital waveforms; and one or more processors configured to execute code that causes the one or more processors to: receive the one or more digital waveforms from the one or more ADCs, each of the one or more digital waveforms corresponding to a set of tuning parameters applied to the DUT; build one or more image tensors of the one or more digital waveforms; use an artificial intelligence embedding model that generates one or more text strings from metadata associated with the waveforms and embeds the metadata and the one or more image tensors into a vector; access a vector database; receive a set of indices from the vector database, the set of indices having a number of indices corresponding to a number of matches; use the set of indices to access an array of best tuning parameters to find one or more sets of best tuning parameters for the DUT; and validate operation of the DUT with the one or more sets of best tuning parameters from the array of best tuning parameters.

2. The test and measurement system of claim 1, wherein the code that causes the one or more processors to validate operation of the DUT comprises code that causes the one or more processors to: tune the DUT with one of the one or more sets of best tuning parameters; test the DUT with the one of the one or more sets of best tuning parameters; determine whether the DUT passes the test; use another of the one or more sets of best tuning parameters when the DUT fails the test; and repeat until one of the following two occurs: the DUT passes the test or all sets of the one or more sets of best tuning parameters have been tested.

3. The test and measurement system of claim 2, wherein the one or more processors are further configured to execute code that causes the one or more processors to employ a user test procedure.

4. The test and measurement system of claim 3, wherein the code that causes the one or more processors to employ a user test procedure comprises code that causes the one or more processors to use a closest set of the one or more sets of best tuning parameters to start the user test procedure.

5. The test and measurement system of claim 3, wherein the code that causes the one or more processors to employ a user test procedure comprises code that causes the one or more processors to update the artificial intelligence embedding model with text strings from metadata associated with the waveforms and the one or more image tensors.

6. The test and measurement system of claim 3, wherein the code that causes the one or more processors to employ a user test procedure includes code that causes the one or more processors to discard or repair the DUT when the DUT cannot be adjusted.

7. The test and measurement system of claim 1, wherein the one or more processors are further configured to execute code that causes the one or more processors to reduce the number of indices using the metadata before using the indices.

8. The test and measurement system of claim 1, wherein the one or more processors are further configured to execute code that causes the one or more processors to read the adjustment parameters from the metadata and perform a regression analysis to determine a most likely value of the adjustment parameters and return a set of regression adjustment parameters.

9. The test and measurement system of claim 8, wherein the one or more processors are further configured to use the set of regression adjustment parameters when the DUT cannot be adjusted by the indices or user test procedure returned from the artificial intelligence vector system.

10. The test and measurement system of claim 1, wherein the one or more processors are further configured to execute code to cause the one or more processors to train the artificial intelligence embedding model.

11. The test and measurement system of claim 1, wherein the code that causes the one or more processors to train the artificial intelligence embedding model includes code that causes one or more processors to: collect adjustment data regarding a predetermined number of DUTs, the adjustment data including optimal adjustment parameter arrays, image tensors, and text; and input the adjustment data to the artificial intelligence embedding model to allow the artificial intelligence embedding model to convert the adjustment data and update the vector database.

12. A method of testing a device under test (DUT), comprising: receiving one or more digital waveforms from one or more analog-to-digital converters (ADCs), each digital waveform corresponding to a set of adjustment parameters applied to the DUT; constructing one or more image tensors of the one or more digital waveforms; using an artificial intelligence embedding model that generates one or more text strings from metadata associated with the waveforms and embeds the metadata and the one or more image tensors; accessing a vector database to compare the vector to vectors in the database; receiving a set of indices from the vector database, the set of indices having a number of indices corresponding to a number of matchers; using the set of indices to access an optimal adjustment parameter array to find one or more optimal sets of adjustment parameters for the DUT; and verifying operation of the DUT with one of the one or more optimal sets of adjustment parameters from the optimal adjustment parameter array.

13. The method of claim 12, wherein verifying operation of the DUT includes: ​ adjusting the DUT with one of the one or more optimal set of adjustment parameters; testing the DUT with the test and measurement instrument with the one of the one or more optimal set of adjustment parameters; determining whether the DUT passed the test; using another of the one or more optimal set of adjustment parameters when the DUT test fails; and repeating until one of the DUT has passed the test or all of the one or more optimal set of adjustment parameters have been tested.

14. The method of claim 13, further comprising employing a user test process.

15. The method of claim 14, wherein employing a user test process comprises using the closest set of the one or more optimal set of adjustment parameters to begin the user test process.

16. The method of claim 14, wherein employing a user test process comprises updating the vector database with a new vector generated from one or more text strings from the metadata associated with the waveform, the one or more image tensors, and the metadata by using the artificial intelligence embedding model to generate the new vector.

17. The method of claim 14, wherein employing a user test process comprises discarding or repairing the DUT when the DUT cannot be adjusted.

18. The method of claim 12, further comprising using the metadata to reduce the number of indices before the one or more processors use the indices.

19. The method of claim 12, further comprising reading the adjustment parameters from the metadata and performing a regression analysis to determine the most likely values of the adjustment parameters and returning a regression adjustment parameter set.

20. The method of claim 12, further comprising using the regression adjustment parameter set when the DUT cannot be adjusted by the indices returned from the vector database or a user test process.

21. The method of claim 12, further comprising training the artificial intelligence embedding model.

22. The test and measurement instrument of claim 21, wherein training the artificial intelligence embedding model comprises: collecting adjustment data on a predetermined number of DUTs, the adjustment data comprising optimal adjustment parameter arrays, tensor images, and text, each forming a vector; and inputting the vectors into the vector database to populate the vector database.

Images