Test and measurement system and method for testing device under test

Abstract

To efficiently update learning of a neural network with new data of a DUT.SOLUTION: A test and measurement system includes: a machine-learning system 246; a test and measurement instrument 220 having a connection part to a device-under-test (DUT) 210; and one or more ADCs for receiving signals from the DUT 210 and converting them into one or more digital waveforms. One or more processors receive one or more digital waveforms corresponding to a set of tuning parameters applied to the DUT 210 and construct one or more image tensors of the one or more digital waveforms. The processors generate one or more text strings from metadata and converts the metadata and the one or more image tensors into an embedded representation vector by using an artificial intelligence embedded representation model, and accesses a vector database. The processors receive a set of indices having indices as many as a number corresponding to the number of matching and use the set to find one or more sets of optimal tuning parameters to verify an operation of the DUT 210.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present disclosure relates to test and measurement systems and methods, and more particularly to applying machine learning to measurements of signals from devices under test (DUTs). [Background technology]

[0002] No. 11,923,896, issued March 5, 2024, entitled "Optical Transmitter Tuning Using Machine Learning" (the "'896 Patent," the contents of which are incorporated herein by reference), and U.S. Patent No. 11,923,895, issued March 5, 2024, entitled "Optical Transmitter Tuning and Reference Parameters Using Machine Learning" (the "'895 Patent," the contents of which are incorporated herein by reference), both describe systems and methods for predicting optimal tuning parameters for an optical transceiver (DUT) using trained machine learning systems. Embodiments of these systems may be generally referred to in this disclosure as OptaML and OptaML Pro.

[0003] Other Tektronix patents and patent applications describe methods for improving the resolution of key features, extending the record length of segments placed within an image, and using other extracted feature sets to more efficiently utilize image space. For example, U.S. patent application Ser. No. 17 / 747,954, filed May 18, 2022, entitled "Machine Learning for Measurements Based on a Short Pattern Waveform Database" (hereinafter referred to as the "'954 Application," the contents of which are incorporated herein by reference), describes a system and method for predicting measurements of signals received from a device under test (DUT) using a trained machine learning system. An example of a signal measurement, in the case of an optical transceiver, is the Transmitter and Dispersion Eye Closure Quaternary (TDECQ). The input to the machine learning system described in the '954 Application, both during training and at run time, is in the form of a tensor image constructed using graphical representations of specific short pattern segments within long waveform data acquired from the DUT.

[0004] As another example, U.S. patent application Ser. No. 18 / 199,846, entitled "Automatic Tuning of Cavity Filters Using Machine Learning," filed May 19, 2023 (hereinafter the "'846 Application," the contents of which are incorporated herein by reference), describes a system and method that utilizes a trained machine learning system to predict optimal positions for tuning elements in a tunable device under test, such as a cavity filter. Input to the machine learning system described in the '846 Application, both during training and at run time, is in the form of tensor images constructed using graphical representations of S-parameters or other parameters that characterize the DUT.

[0005] As another example, U.S. Patent No. 18 / 665,258, filed November 28, 2024, entitled "User Interface for a Tensor Builder for Constructing Images for Input to Machine Learning" (hereinafter the "'258 Application"), the contents of which are incorporated herein by reference, describes a system and method for allowing a user to select a tensor format and constructing tensors of the selected format from data from a DUT. The tensors are then sent to a machine learning network. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] U.S. Patent No. 1,1923,896 [Patent Document 2] US Patent Application Publication No. 2022 / 0373598 [Patent Document 3] US Patent Application Publication No. 2023 / 0398694 [Patent Document 4] US Patent Application Publication No. 2024 / 0393918 Summary of the Invention [Problem to be solved by the invention]

[0007] In conventional systems, the device could be tuned by user processing, but the device could not predict new data, and there was no efficient way to update the neural network training with new device data. Updating these conventional systems required time-consuming and tedious retraining of the neural network. Furthermore, conventional systems have the disadvantage that they are only trained for one model or type of DUT with a specific number of tuning parameters. If a different model of DUT with a different number of tuning parameters is used, the system must be retrained.

[0008] There is a need for systems that can work with any DUT with any number of tuning parameters, and users should be able to update these systems to tune new types of DUTs or add new DUT tuning data without having to train or retrain neural networks. [Means for solving the problem]

[0009] Embodiments of the present application include artificial intelligence (AI) for embedding waveform image tensors into vector representations within a vector database system, eliminating the need for neural network training. These embodiments include a combination of vision and text transformers within a pre-trained neural network and vector database. These embodiments rely on tensor construction and feature extraction to support specific models of the device under test (DUT). These capabilities are implemented using Tektronix's OptaML TM Because they exist within the system, users of the system can integrate these embodiments into their systems. Additionally, these embodiments include a new regression classifier that can also be integrated into existing systems. These functions will be available to new users using the embodiments disclosed herein.

[0010] This embodiment employs a vector database system that allows users to store and retrieve vectors of complex information. This particular implementation is not limited to the use of a particular vector database system, nor is the module used to associate images with text descriptions limited to a particular module; information is typically communicated between the module and the database external to the vector database system.

[0011] However, for ease of understanding, the present embodiment uses a subscription-based vector database called Pinecone. The present embodiment also uses CLIP. Meanwhile, Pinecone provides an application programming interface (API) to several image-text pairing embedding models, including OpenAI's CLIP. CLIP (Contrastive Language Image Pretraining) can associate images with text descriptors using a combination of image recognition and natural language processing. In this description, other terms have specific meanings. As used herein, "LLM" refers to Large Language Model. "ML" refers to machine learning. "MFG" refers to manufacturing. "RAG" refers to search augmentation generation, a means of providing additional information to LLM. The term "tensor" refers to an image containing extracted waveform features and metadata. This represents the characteristics of the DUT and how its tuning parameters and metadata affect the waveform.

[0012] The primary purpose of the system described in the '895 and '896 patents is tuning DUT parameters on the production line. For one-shot tuning prediction, the process feeds tensor images from the current DUT into a neural network and retrieves a set of tuning parameter numbers for direct storage in the DUT's registers. This eliminates the need for the user to go through a tuning procedure in which the tuning process sweeps through all tuning parameters until the optimal setting is found. This results in significantly reduced tuning time on the production line. [Brief explanation of the drawings]

[0013] [Figure 1] Figure 1 shows an RGB image of the waveform tensor. [Figure 2A]FIG. 2A shows a block diagram of an embodiment of a machine learning system during training. [Figure 2B] FIG. 2B shows a block diagram of an embodiment of a machine learning system at runtime. [Figure 3] FIG. 3 shows a block diagram of one embodiment of an AI vector embedded representation system. [Figure 4A] Figure 4A shows an embodiment of an AI vector embedded representation system illustrating vector-database interactions and system behavior. [Figure 4B] Figure 4B shows an embodiment of the AI ​​vector embedded representation system, illustrating vector-database interactions and system behavior. [Figure 5] FIG. 5 shows an embodiment of a test and measurement device. DETAILED DESCRIPTION OF THE INVENTION

[0014] Figure 1 shows an embodiment of an RGB image 10 of a waveform tensor. This RGB (red, green, blue) image contains segments of three waveforms 12, each representing a different one of three tuning parameter sets. The image may also include bar graphs (e.g., 14 and 16) representing additional information such as temperature or noise, and short pattern segments (e.g., 18) based on these waveforms with multiple instances overlaid.

[0015] 2A and 2B are block diagrams illustrating the training and runtime operations of an example conventional machine learning system. In FIG. 2A, neural network 26, which in one embodiment is a Resnet18 neural network, is trained by receiving image tensors containing waveform segments and other metadata (e.g., temperature, noise, etc.). In one example, a user collects waveforms and data from several devices under test (DUTs) at 20. For example, the user may collect waveforms and data for 200 DUTs and create an array of 200 image tensors at 22. The user also tunes each DUT at 24, creating an array of 200 optimal tunings to match each DUT's image tensor. These two arrays are provided to neural network 26, and a training operation adapts the neural network to associate image tensors with the optimal tuning sets. In one embodiment, the outer three layers of Resnet18 are trained to associate image tensors with the optimal tuning set. The remaining layers of the pre-trained Resnet18 network are left unchanged in the transfer learning process.

[0016] Figure 2B shows the run-time operation of a trained system, where the neural network has learned a specific model of a DUT with a specific number of tuning parameters. During the run-time of these conventional systems, the neural network 26 may be used to make tuning predictions for each DUT during the manufacturing process. The user loads the DUT to be tuned using the same three sets of reference parameters used during training and collects data for that DUT at 28. The tensor builder 22 then constructs a tensor image using the three waveforms acquired from the DUT and, possibly, other information contained in a bar graph or other display. The neural network 26 receives the resulting tensors. The neural network outputs the optimal set of tuning parameters, which are register values ​​stored in the DUT's registers. The user can then verify the tuning. If the DUT passes, the user moves on to the next DUT. If the DUT fails, the user attempts tuning using the original slow algorithm to make a final pass / fail decision.

[0017] Embodiments of the present application generally use AI embeddings in a vector database to achieve the advantage of eliminating the need to train a neural network to add new DUT models or add new data to existing models. AI embedding models may require pre-training, in which case pre-training may be performed. Pre-training is not required if the text and image embedding vectors are sufficiently diverse and unique for every example. The need for pre-training depends on the ability of the embedding model to adequately represent diversity in the tensors, which is important in relation to tuning parameters.

[0018] Embodiments of the present application generally employ similar techniques for DUT characterization using three or more reference parameter sets, as described, for example, in the '895 patent or the '846 application. Like the '895 patent, embodiments of the present disclosure also employ a similar tensor builder to place three or more acquired waveforms into an image tensor. However, some embodiments of the present disclosure may also employ embedded representations of textual language associated with the DUT for enhanced capabilities.

[0019] The present embodiment differs from previous systems by using an AI embedded representation model and vector database. Previous systems still typically use pre-trained neural networks, such as Resnet18. However, the present neural network is not trained on directly output tuning parameters, as is done in previous systems such as those shown in Figures 2A and 2B. Instead, the present neural network is trained to output vectors in a multidimensional space. Image tensors are converted into embedded representation vectors, which are stored and indexed in the vector database. This mechanism converts the deep learning network into components that can be used for any new DUT training data or new DUT model formats.

[0020] As described above, embodiments of the present disclosure employ a CLIP embedding model. Other types of embedding models, such as BLIP-2, FLIP, and SigLIP, perform similar functions for image-text pairings. These models accept images and text strings as input, convert them into vectors, and store them in a vector database. There are various types of AI embedding models available, each specialized for different data types, such as language, audio, images, and video. Some embodiments of the present disclosure use a CLIP model incorporating Resnet18, a pre-trained deep learning network that performs well at extracting low-level features from waveform tensors. However, embodiments are not limited to the use of a CLIP model. Networks such as the CLIP model can extract key features of waveforms that are best related to tuning parameters. The CLIP model also includes a Vision Transformer (ViT), which may provide additional benefits. Furthermore, the model includes a language transformer that can embed language associated with an image into an embedding vector.

[0021] Additional text may be used to assist vector matching by including metadata such as the temperature of the DUT, measurement noise from the waveform that was filtered out for feature extraction, the DUT model number, and other relevant information, which allows the system to separate the vector database match search into a smaller set of vectors, speeding up the search process and reducing the cost of subscription services or other vector database applications that may be used with embodiments of the disclosed technology.

[0022] AI embedded representation models may require pre-training to better address expected input tensor image and text types. This pre-training furthers the goal of deploying these AI embedded representation models to work with a wide range of DUTs that generate similar types of tensor images. Metadata text, including measurements, model numbers, and other metadata, enhances the feasibility of this goal, as it is an important part of reducing the set of vectors to search for tuning matches. This mechanism partially offsets the burden of identifying the image resolution for tensor images and Resnet18 neural networks. This allows the neural network to operate with a wider variety of DUT types and conditions, making the system more generalizable and able to handle a wider range of use cases without retraining.

[0023] Figure 3 shows a block diagram of the test and measurement system for both training and runtime. The left side of the diagram shows the system in pre-training 30. The right side shows the system in runtime 60. While the system components are the same and are shown twice for clarity, the two processes, pre-training and runtime, are connected when the runtime system does not provide the DUT with the parameters necessary to pass the test. These system components may have different reference numbers between the two processes if they result in or originate from different processes.

[0024] An external computing device 49 is shown as a bridge between the two. This computing device may comprise a manufacturing computer that runs the user test process and interacts with the test and measurement equipment 72 that tests the DUT in FIG. 2B, the vector database 44, and the AI ​​embedded representation model 40. The AI ​​embedded representation model may consist of programs executed by one or more processors on the external computing device or may consist of programs running on a remote server. The vector database may operate as a local database on the computing device or may operate on a separate server. The test and measurement system may have one or more processors anywhere in the system that execute programs to perform various operations and tasks.

[0025] A primary starting point for pre-training the system is collecting data from a large number of DUTs. In some embodiments, data is collected from approximately 200 nominal DUTs at 32. This process has many similarities to traditional OptaML systems. The system of FIG. 3 also determines several reference parameter sets to load into all DUTs, collects waveforms from each set of DUTs, extracts features from these waveforms, and sends them to tensor images 34 and language text 38. This process produces arrays of tensor images and text representing the variations of the group of DUTs. This process also includes optimally tuning these sample DUTs at 36 and saving the optimal tuning sets to an array of optimal tuning parameter sets 46.

[0026] Unlike conventional systems, the system of the present embodiment does not use an array of optimal tuning parameters and an array of tensor images to train the neural network. Instead, the tuning arrays are saved for later reference to retrieve the optimal tuning set. The array of tensor images is input into the CLIP AI embedded representation model and converted into an indexed array of vectors. These vectors are stored in a vector database, and the indexes associated with these vectors are the same as the indexes of the associated arrays of tuning parameters 46. Once the vector database contains a set of vectors representing many different DUTs and the associated arrays of optimal tuning parameters have been created, the system is ready to be used to tune new DUTs at runtime on the manufacturing line.

[0027] Once the system is trained (i.e., the vector database is populated with a set of embedded representation vectors from the AI ​​embedded representation model), a user can begin using the system during runtime 60. The user then takes a single device and collects waveforms from this DUT at 62, understanding that some manufacturing lines may have multiple DUTs running in parallel. The tensor builder 34 builds tensors, the embedded representation model 40 and neural network 42 operate as before to provide text, and the text transformer 38 and vision transformer 48 (if used) generate AI embedded representation vectors. The vector database 44 then runs a vector matching process 64 to generate a set of vectors containing the DUT's optimal tuning parameters at 66. The tuning parameters are extracted from the vectors and set into the DUT at 70. This process then tests the DUT at 72. If the DUT passes at 74, the system returns to runtime and proceeds to test the next DUT.

[0028] If the DUT fails one of the tuning parameter sets, the process may be repeated. Unlike conventional systems, the vector matching process generates several vectors, so that the runtime side generates several sets of tuning parameters that closely match the optimal tuning parameters from the array of optimal tuning parameters 66. The iteration process includes tuning other tuning parameters than those placed and tested on the DUT. If all tuning parameter sets fail at 74, the process tunes the DUT at 76. The DUT is tuned by the user's conventional process at 76. If the DUT still fails at this point, it is discarded or sent for repair. If the DUT is tuned, the image tensor constructed for this DUT is retrieved from the tensor builder at runtime, and that information is used to generate a new embedding vector using the image tensor from the runtime side, also updated with the tuning parameters associated with the text if successful. This is then used to update the vector database. This ensures that the database continues to grow and improve over time as new DUT models are added and performance optimized. The iterative process of tuning the database and adding new models improves the accuracy and efficiency of the system, ultimately leading to better results and increased productivity.

[0029] However, once a DUT is successfully tuned, a new tensor image and set of optimal tuning parameters are added to the vector database and optimal tuning array, respectively. This ensures that the database continues to grow and improve over time as new DUT models are added and performance optimized. This iterative process of tuning the database and adding new models improves the accuracy and efficiency of the system, ultimately leading to better results and increased productivity.

[0030] The vector database 50 may be comprised of the Pinecone database, a vector database subscription service. This particular vector database offers a wide range of features for managing and expanding the database as it is used. Currently, it is a good option for implementing programs (code) with minimal effort, as it provides high-level commands that are very easy to use. Pinecone also supports various types of AI embedding representation models, such as CLIP, that can be specified to create vector embedding representations. However, embodiments of the disclosed technology are not limited to using Pinecone.

[0031] In addition to supporting various AI embedded representation models, Pinecone also provides multiple types of queries and data retrieval from the vector database. Embodiments of the present application generally use image-to-image queries with the CLIP AI embedded representation model to convert images to vectors. These vectors are used to access the optimal tuning parameter array and retrieve the optimal tuning parameters. This process allows the system to identify the optimal tuning parameters for a new DUT by comparing the vectorized image of the new DUT with images of the DUT model stored in the database, thereby optimally tuning the new DUT.

[0032] Even if the best-matching tuning is not optimal, it will likely be very close to the optimal tuning, allowing the user algorithm to sweep the tuning from a close starting point. As a result, this sweep tuning process takes less time than starting from the typical default state. This is because the best-matching tuning provides a good starting point for the user algorithm, reducing the time and effort required to sweep tune the DUT. By using the vector database and the best-matching tuning as a starting point, the system significantly improves the efficiency and accuracy of the tuning process, ultimately leading to better results and increased productivity.

[0033] As the vector database grows and more DUT models are added and tuned, it becomes increasingly likely that one-shot tuning will be sufficient for each new DUT. This means that the number of times a user tuning algorithm will be needed decreases over time as the database becomes more comprehensive and accurate.

[0034] If the AI ​​embedded representation model does not produce the required resolution for image matches / distinctions, it can be retrained once to better accommodate a greater number of variations in the input. However, this training is expected to be a one-time event; during runtime, as new data is added to the vector database, no additional training is necessary. Retraining the AI ​​embedded representation model allows the system to improve its accuracy and efficiency in identifying optimal tuning parameters for new DUTs. However, this retraining is expected to be necessary only in rare cases; the system will continue to operate reliably and accurately with the existing vector database and AI embedded representation model. The ultimate goal is to build a system that is self-contained and requires minimal maintenance and tuning, allowing engineers to focus on other aspects of their work without worrying about the tuning process.

[0035] Comparing Figures 2A and 2B with Figure 3 highlights the differences between the prior art tuning system and the new tuning system. The key difference is that in the prior art system, the neural network learns a fixed set of labels and cannot add new labels without retraining. In contrast, the new system uses a neural network that outputs embedding vectors of common images stored in a vector database. This allows new labels and DUT models to be added without the need to retrain the neural network. The optimal tuning set results from matching input images with existing images in the vector database. This approach allows the system to quickly and accurately identify optimal tuning parameters for each new DUT model type added to the system without requiring extensive training and retraining of the neural network. Overall, the new tuning system is more flexible, efficient, and scalable than the prior art system, making it easier for engineers to optimize the performance of new DUT models and improve overall productivity.

[0036] Figures 4A and 4B show an embodiment of the overall system in more detail regarding the interactions between the various parts. The two figures are drawn in a side-by-side arrangement to show the connections between all of the system's components. The overall system includes a tensor builder 34, an AI embedded representation model 40, a vector database 50, and a post-processing classifier module 110. The system operates in either an append-new-data mode or a runtime mode, as shown at 80. Thus, data consists of either data for a new DUT not previously present in the vector database or data for the current DUT under test. The tensor builder 34 places three or more acquired waveforms into an image tensor. The tensor builder may extract S-parameters 82, short-pattern waveforms 84, DSP transforms 86 (fast Fourier transform, bispectral transform, filtering, correlation, or other), and measurements 88. Measurements include frequency, amplitude, pulse width, transmitter and dispersion eye closure quaternary (TDECQ), noise, and temperature, all of which are traditionally included in image tensors. Additionally, DUT metadata may be collected, including model, vendor, and type. The tensor builder creates an RGB image tensor at 90, allowing different information to be placed in different color channels of the image. The tensor builder also creates text strings of various pieces of information at 91, which are passed to the AI ​​embedded representation model (e.g., CLIP) described above. The text strings are also sent to the vector database shown in Figure 4B.

[0037] The AI ​​embedded representation model 40 takes the RGB image and uses a neural network 42, possibly in combination with a vision transformer 48, to create part of the combined vector. The text transformer / encoder 38 creates the other part, resulting in a multidimensional spatial embedded representation combined vector, which is passed to the vector database 50 of Figure 4B.

[0038] In Figure 4A, switches 93 and 95 are in the "Neural Network" position, selecting between the Neural Network and the Vision Transformer.

[0039] In Figure 4B, the vector database receives the combination vectors from database 50, the text strings from text string generation block 91, and the data collected from the DUT 32, as shown in Figure 4A. The combination vectors are written to the database at 94, along with measurement metadata 96, other classification metadata 98, and tuning parameter metadata 100. This data is also passed to a metadata acquisition block at 104. The vector database 50 performs vector matching at 64, with the goal of finding a set of N best matches at 92. The set of best matches 92 is then passed to a classifier 110. Additionally, the vector database filters the metadata at 102 to improve vector matching results, speed up matching, and reduce the time required to search for vectors.

[0040] The classifier 110 then obtains the vector IDs of the set of N best matches. The processing algorithm 106 operates to reduce the set of matches at 108. This block receives the embedded representation IDs of the N best matches of the DUT from the vector database. The best matches are for new DUTs to be tuned on the production line. Some waveform or other type of characterization data is acquired from the DUT using reference parameters stored in the DUT. Next, features and measurements are extracted from the waveforms and incorporated into image and text tensors and metadata to obtain an embedded representation object set, which is then compared to the embedded representations in the database. This is done using the metadata in the embedded representation vector ID structure to further classify the vectors into classification groups. The interpolation block 118 reads the tuning parameters stored in the metadata set of embedded representation IDs, performs a regression analysis for each tuning parameter, and calculates the most likely value of the tuning parameter. This is one of the tuning parameter sets returned to the user to try on their DUT. Other tuning parameter sets are also returned to the user to try with their DUT. The user may try these various sets with their DUT to increase the chances of finding a tuning parameter set that meets the user's requirements for tuning the DUT.

[0041] As mentioned above, if the best match is not satisfactory, the system may try several suboptimal matches to obtain better results. Additionally, the user may choose to try the returned regression pseudo-interpolated tuning parameter sets. If one of these matches is determined to be optimal, the system proceeds to tune the next DUT. This iterative process validates and optimizes the selected tuning parameters for each DUT for the best possible performance.

[0042] Switches 97 and 99 in Figure 4B show two operational states based on the switch control signal from block 120. The first state is "new data" and the second state is "runtime." The runtime state is when tuning a new DUT. Its AI embedding is compared to the embeddings in the database, and the N IDs of the best matching vectors are returned. These IDs are processed by a regression and classification block to reduce the set and further classify for accuracy. The best matches are used to obtain the optimal tuning parameters from the ID metadata and then to obtain the optical tuning parameters. In the runtime state, switch 99 is normally closed (NC).

[0043] Selecting "New Data" in block 120 changes switches 97 and 99 to their respective "New Data" settings. Block 122 receives new data collected from the DUT from block 32 of FIG. 4A via switch 99. Additionally, switch 97 transitions to NC, which disconnects the classifier from the vector database. Data can be added to the vector database without retraining the neural network.

[0044] The resulting system becomes more accurate at making predictions about data it has never seen before. This architecture includes text in addition to images as part of the vectors. This takes a large part of the burden off the neural network's image interpretation. This text metadata is also used to select which folder the vector goes to in the database. This also speeds up vector matching operations, as the search is performed on a much smaller set of vectors.

[0045] Once the system "understands" waveform tensors better, the goal is to have it work with all future calibration setups using the tensor builder image, possibly with a one-time training step if necessary. This goal would be reasonable if each different tensor image / text embedding representation was unique and different enough to be detected by a cosine matching algorithm. All output predictions are a valid, optimal set of tunings, since they are derived from an array of optimal tunings rather than potentially invalid outputs of the neural network.

[0046] Use cases and applications of embodiments of the present application include, by way of example only, one or more of the following: One use case is tuning optical transmitters on a production line, saving significant production time; Another use case is tuning nonlinear correction parameters or frequency response calibration parameters for oscilloscopes in production, saving time; A data reduction embodiment is setting up sweeps to test die before they are assembled into a package, saving production time; Another use case is tuning diplexers and duplexers for wireless communication antennas, using S-parameters as input for tensor images.

[0047] FIG. 5 illustrates an embodiment of a test and measurement instrument 220 for performing performance measurements on a DUT (such as 210) in conjunction with a machine learning system 246. The test and measurement instrument generally interfaces with the DUT 210 via a probe 232. The test and measurement instrument 220's acquisition (waveform data acquisition) circuitry 236 may include an analog-to-digital converter (ADC) to digitize the input signal, a clock recovery circuit, and trigger hardware to provide timing. A processor 238 may control the acquisition hardware and the rendering of the input signal into a waveform. A display 242 displays the acquired waveform to a user. A user interface device 244 may include optional touch screen functionality on the display, allowing a user to interact with the test and measurement instrument 220.

[0048] Memory 240 enables processor 238 to store and process waveform data and may store executable programs (code). The overall system, including test and measurement instrument 220, may include one or more processors configured to execute programs that cause the one or more processors to perform the various tasks described herein. The one or more processors may include one or more processors on test and measurement instrument 220 and one or more processors on machine learning system 246. Machine learning system 246 may be a separate computing device that receives data from test and measurement instrument 220. A separate database structure 248 may store the entire waveform database or may form part of machine learning system 246.

[0049] Aspects of the disclosed technology may operate on specially created hardware, firmware, digital signal processors, or specially programmed general-purpose computers, including processors that operate according to programmed instructions. The terms "controller" or "processor" herein contemplate microprocessors, microcomputers, ASICs, and dedicated hardware controllers, among others. Aspects of the disclosed technology may be embodied in computer-usable data and computer-executable instructions, such as one or more program modules, executed by one or more computers (including a monitoring module) or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc., which, when executed by a processor in a computer or other device, perform particular tasks or implement particular abstract data types. Computer-executable instructions may be stored in computer-readable storage media, such as hard disks, optical disks, removable storage media, solid-state memory, RAM, etc. Those skilled in the art will appreciate that the functionality of the program modules may be combined or distributed as desired in various embodiments. Furthermore, such functionality may be embodied in whole or in part in firmware or hardware equivalents, such as integrated circuits, field programmable gate arrays (FPGAs), etc. Certain data structures may be used to more effectively implement one or more aspects of the disclosed technology, and such data structures are considered within the scope of the computer-executable instructions and computer-usable data described herein.

[0050] The disclosed aspects may, in some cases, be implemented in hardware, firmware, software, or any combination thereof. The disclosed aspects may also be implemented as instructions carried by or stored on one or more computer-readable media, which may be read and executed by one or more processors. Such instructions may be referred to as a computer program product. As used herein, computer-readable media refers to any medium that can be accessed by a computing device. By way of example and not limitation, computer-readable media may include computer storage media and communication media.

[0051] "Computer storage media" means any medium that can be used to store computer-readable information. By way of example and not limitation, computer storage media may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory and other memory technologies, compact disc read-only memory (CD-ROM), digital video disc (DVD) and other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage and other magnetic storage devices, and any other volatile or nonvolatile, removable or non-removable medium implemented in any technology. "Computer storage media" excludes signals themselves and transitory forms of signal transmission.

[0052] A communication medium means any medium usable for communicating computer-readable information. By way of example, and not limitation, communication media may include coaxial cable, fiber optic cable, air, or any other medium suitable for communicating electrical, optical, radio frequency (RF), infrared, acoustic, or other types of signals. Example

[0053] The following examples are provided to aid in understanding the technology disclosed in this application. Embodiments of the technology may include one or more of the examples described below, and any combination thereof.

[0054] Example 1 is a test measurement system, a connection for enabling the test and measurement equipment to be connected to a device under test (DUT); one or more analog-to-digital converters (ADCs) that receive signals from the DUT and convert them into one or more digital waveforms; the test and measurement device having one or more processors, the one or more processors receiving the one or more digital waveforms from the one or more ADCs, each corresponding to a set of tuning parameters applied to the DUT; constructing one or more image tensors of said one or more digital waveforms; generating one or more text strings from metadata associated with the one or more digital waveforms, and using an artificial intelligence embedded representation model to convert the metadata and the one or more image tensors into embedded representation vectors; Processing to access the vector database; receiving a set of indices from the vector database, the set having a number of indices corresponding to the number of matches; accessing an array of optimal tuning parameters using said set of indices to find one or more sets of optimal tuning parameters for said DUT; verifying operation of the DUT using the one or more sets of optimal tuning parameters from the array of optimal tuning parameters; The one or more processors are configured to execute a program that causes the one or more processors to perform the steps of:

[0055] A second embodiment is the test and measurement system of the first embodiment, wherein the program causing the one or more processors to perform a process of verifying the operation of the DUT comprises: tuning the DUT with one of the one or more optimal tuning parameter sets; testing the DUT with the one of the one or more optimal tuning parameter sets; determining whether the DUT passed the test; using an alternative set of the one or more optimal tuning parameters if the DUT fails the test; repeating the process until either the DUT passes the test or all of the one or more sets of optimal tuning parameters have been tested. The program includes a program that causes one or more processors to perform the above.

[0056] Example 3 is the test and measurement system of Example 2, wherein the one or more processors are further configured to execute a program that causes the one or more processors to perform processing utilizing a user test process.

[0057] Example 4 is a test and measurement system of Example 3, wherein the program that causes the one or more processors to perform processing that utilizes a user test process includes a program that causes the one or more processors to perform processing that starts the user test process using a best set of the one or more optimal tuning parameter sets.

[0058] Example 5 is the test and measurement system of Example 3, wherein the program that causes the one or more processors to perform processing utilizing a user test process includes a program that causes the one or more processors to perform processing to update the artificial intelligence embedded representation model with text strings derived from metadata associated with the one or more waveforms and the one or more image tensors.

[0059] Example 6 is a test and measurement system of Example 3, wherein the program that causes the one or more processors to perform processing that utilizes a user test process includes a program that causes the one or more processors to perform processing to discard or repair the DUT if the DUT cannot be tuned.

[0060] Example 7 is the test and measurement system of any of Examples 1 to 6, wherein the one or more processors are further configured to execute a program that causes the one or more processors to perform a process of reducing the number of indexes using the metadata before the one or more processors use the indexes.

[0061] Example 8 is a test and measurement system of any of Examples 1 to 7, wherein the one or more processors are further configured to execute a program that causes the one or more processors to read tuning parameters from the metadata, perform a regression analysis to determine most likely values ​​for the tuning parameters, and return a set of regression tuning parameters.

[0062] Example 9 is the test and measurement system of Example 8, wherein the one or more processors are further configured to use the regression tuning parameter set if the DUT cannot be tuned by either an index returned from the vector database or a user test process.

[0063] Example 10 is the test and measurement system of any of Examples 1 to 9, wherein the one or more processors are further configured to execute a program that causes the one or more processors to perform a process of training the artificial intelligence embedded representation model.

[0064] An eleventh embodiment is the test measurement system according to any one of the first to tenth embodiments, wherein the program causing the one or more processors to perform a process of learning the artificial intelligence embedded expression model comprises: collecting tuning data for a predetermined number of DUTs, including optimal tuning parameter arrays, tensor images, and text; inputting the tuning data into the artificial intelligence embedded representation model, allowing the artificial intelligence embedded representation model to convert the tuning data and update the vector database; The program includes a program that causes one or more processors to perform the above.

[0065] Example 12 is a method for testing a device under test (DUT), comprising: receiving one or more digital waveforms from one or more analog-to-digital converters (ADCs), each corresponding to a set of tuning parameters to be applied to the DUT; constructing one or more image tensors of said one or more digital waveforms; generating one or more text strings from metadata associated with the one or more digital waveforms, and using an artificial intelligence embedded representation model to convert the metadata and the one or more image tensors into embedded representation vectors; accessing a vector database and comparing the embedded representation vector with vectors in the vector database; receiving a set of indices from the vector database, the set having a number of indices corresponding to the number of matches; accessing an array of optimal tuning parameters using said set of indices to find one or more sets of optimal tuning parameters for said DUT; verifying operation of the DUT using one of the one or more optimal tuning parameter sets from an array of optimal tuning parameters; It is equipped with.

[0066] Example 13 is the method according to claim 12, wherein the process of verifying the operation of the DUT comprises: tuning the DUT with one of the one or more sets of optimal tuning parameters; testing the DUT with the one of the one or more sets of optimal tuning parameters using a test and measurement instrument; determining whether the DUT passed the test; using an alternative set of the one or more optimal tuning parameter sets if the DUT fails the test; iterating the process until either the DUT passes the test or has been tested with all of the one or more sets of optimal tuning parameters; It has.

[0067] Example 14 is the method of example 13, further comprising utilizing a user testing process.

[0068] Example 15 is the method of Example 14, wherein the process of utilizing a user testing process includes a process of initiating the user testing process using a best one of the set of one or more optimal tuning parameters.

[0069] Example 16 is the method of example 14, wherein the process utilizing a user testing process includes updating the vector database with the new vectors by using an artificial intelligence embedded representation model to generate new vectors from one or more text strings derived from metadata associated with the one or more digital waveforms, the one or more image tensors, and the metadata.

[0070] Example 17 is the method of example 14, wherein utilizing a user test process includes discarding or repairing the DUT if the DUT cannot be tuned.

[0071] Example 18 is the method of any of Examples 12-17, further comprising: using the metadata to reduce the number of indexes before one or more processors use the indexes.

[0072] Example 19 is a method of any of Examples 12 to 18, further comprising: reading tuning parameters from the metadata; performing a regression analysis to determine most likely values ​​for the tuning parameters; and returning a set of regression tuning parameters.

[0073] Example 20 is the method of example 19, further comprising using the regression tuning parameter set if the DUT cannot be tuned by either the index returned from the vector database or a user test process.

[0074] A twenty-first embodiment is the method according to any one of the twelfth to twentieth embodiments, further comprising the step of training the artificial intelligence embedded expression model.

[0075] Example 22 is the method of Example 21, wherein the process of training the artificial intelligence embedded expression model includes: collecting tuning data for a predetermined number of DUTs, including optimal tuning parameter arrays, tensor images, and text-converted vectors; inputting the vector into the vector database and storing it in the vector database; It has.

[0076] Although the above-described versions of the presently disclosed subject matter have many advantages that have been described or that will be apparent to those skilled in the art, not all of these advantages or features are required in every version of the disclosed devices, systems, or methods.

[0077] Additionally, the description of this application refers to specific features. It should be understood that the disclosure herein includes all possible combinations of these specific features. When a specific feature is disclosed in connection with a particular aspect or example, that feature can also be used in connection with other aspects and examples, to the extent possible.

[0078] Furthermore, when this application refers to a method having two or more defined steps or processes, these defined steps or processes may be performed in any order or simultaneously, unless the circumstances do not preclude this possibility.

[0079] All features disclosed in the specification, claims, abstract and drawings, and all steps in any disclosed method or process, may be combined in any combination, except where at least some of such features or steps are mutually exclusive combinations. Each feature disclosed in the specification, abstract, claims and drawings may be replaced by an alternative feature serving the same, equivalent or similar purpose, unless expressly stated otherwise.

[0080] While specific embodiments of the disclosed technology have been illustrated and described for purposes of illustration, it will be appreciated that various modifications can be made therein without departing from the spirit and scope of the invention. Accordingly, the disclosed technology should not be limited, except as by the appended claims. [Explanation of symbols]

[0081] 22 Tensor Builder 26 Neural Networks 32 New Data Collection Blocks 34 Tensor Builder 38 Text Transformers / Encoders 40 AI embedded expression models 41 Combiner 42 Neural Networks 44 Vector Database 46 Array of optimal tuning parameter sets 48 Vision Transformer 49 External Computing Devices 50 Vector Databases 64 Vector Matching Processing 66 Optimal Tuning Parameter Arrays 96 Measurement Metadata 97 Switch 98 Other Classification Metadata 99 Switch 100 Tuning Parameters Metadata 106 Processing Algorithm 110 Classifier 118 Interpolation Blocks 120 New Data / Runtime Selection Block 122 Block for tuning the DUT for learning

Claims

1. 1. A test and measurement system comprising: a connection that allows the test and measurement equipment to be connected to a device under test (DUT); one or more analog-to-digital converters (ADCs) that receive signals from the DUT and convert them into one or more digital waveforms; the test and measurement device having one or more processors, the one or more processors receiving the one or more digital waveforms from the one or more ADCs, each corresponding to a set of tuning parameters applied to the DUT; constructing one or more image tensors of the one or more digital waveforms; generating one or more text strings from metadata associated with the one or more digital waveforms and using an artificial intelligence embedded representation model to convert the metadata and the one or more image tensors into embedded representation vectors; a process for accessing a vector database; receiving a set of indexes from the vector database, the set having a number of indexes corresponding to the number of matches; accessing an array of optimal tuning parameters using the set of indices to find one or more sets of optimal tuning parameters for the DUT; verifying operation of the DUT using the set of one or more optimal tuning parameters from the array of optimal tuning parameters; a test and measurement system configured to execute a program that causes the one or more processors to perform the steps of:

2. a program for causing the one or more processors to perform a process for verifying the operation of the DUT, tuning the DUT with a set of the one or more optimal tuning parameter sets; testing the DUT with the one of the one or more sets of optimal tuning parameters; determining whether the DUT passed the test; if the DUT fails the test, using another set of the one or more optimal tuning parameters; iterating until either the DUT passes the test or all of the one or more sets of optimal tuning parameters have been tested.

2. The test and measurement system of claim 1, further comprising a program that causes said one or more processors to:

3. 3. The test and measurement system of claim 2, wherein the one or more processors are further configured to execute a program that causes the one or more processors to utilize a user test process to initiate the user test process using a best set of the one or more optimal tuning parameters.

4. 3. The test and measurement system of claim 2, wherein the one or more processors are further configured to execute a program that causes the one or more processors to process, using a user test process, updating the artificial intelligence embedded representation model with text strings and the one or more image tensors derived from metadata associated with the one or more waveforms.

5. 2. The test and measurement system of claim 1, wherein the one or more processors are further configured to execute a program that causes the one or more processors to perform a process of reducing the number of indexes using the metadata before the one or more processors use the indexes.

6. 10. The test and measurement system of claim 1, wherein the one or more processors are further configured to execute a program that causes the one or more processors to read tuning parameters from the metadata, perform a regression analysis to determine most likely values ​​for the tuning parameters, and return a set of regression tuning parameters.

7. The one or more processors: collecting tuning data including optimal tuning parameter arrays, tensor images, and text for a predetermined number of DUTs; inputting the tuning data into the artificial intelligence embedded representation model, allowing the artificial intelligence embedded representation model to transform the tuning data and update the vector database; 2. The test and measurement system of claim 1, further configured to execute a program that causes the one or more processors to train the artificial intelligence embedded representation model by:

8. 1. A method for testing a device under test (DUT), comprising: receiving one or more digital waveforms from one or more analog-to-digital converters (ADCs), each corresponding to a set of tuning parameters to be applied to the DUT; constructing one or more image tensors of the one or more digital waveforms; generating one or more text strings from metadata associated with the one or more digital waveforms and using an artificial intelligence embedded representation model to convert the metadata and the one or more image tensors into embedded representation vectors; accessing a vector database and comparing the embedded representation vector with vectors in the vector database; receiving a set of indexes from the vector database, the set having a number of indexes corresponding to the number of matches; accessing an array of optimal tuning parameters using the set of indices to find one or more sets of optimal tuning parameters for the DUT; verifying operation of the DUT using one of the one or more sets of optimal tuning parameters from an array of optimal tuning parameters; A method for testing a device under test, comprising:

9. The process of verifying the operation of the DUT is as follows: tuning the DUT with a set of the one or more sets of optimal tuning parameters; testing the DUT with the one of the one or more sets of optimal tuning parameters using a test and measurement instrument; determining whether the DUT passed the test; if the DUT fails the test, using another set of the one or more optimal tuning parameters; iterating until the DUT either passes the test or has been tested with all of the one or more sets of optimal tuning parameters; 9. The method for testing a device under test according to claim 8, comprising:

10. reading tuning parameters from the metadata; performing a regression analysis to determine the most likely values ​​for the tuning parameters and returning a set of regression tuning parameters; using the regression tuning parameter set if the DUT cannot be tuned by either the index returned from the vector database or a user test process; 9. The method for testing a device under test according to claim 8, further comprising:

11. collecting tuning data for a predetermined number of DUTs, including arrays of optimal tuning parameters, tensor images, and vectors converted from text; inputting the vector into the vector database and storing the vector in the vector database; 9. The method for testing a device under test according to claim 8, further comprising the step of training the artificial intelligence embedded representation model by:

Images