Test and measurement devices, systems, and methods associated with augmented reality

Abstract

A test and measurement system can include a data store configured to store enhancement settings for dynamically enhancing a physical test environment and a computing device coupled to the data store. The computing device can be configured to receive an input feed from the physical test environment, create an enhanced image based on the enhancement settings and the input feed, and output the enhanced image to be superimposed on the physical test environment to enhance a user's view of the physical test environment.

Description

[0001] Cross - Reference to Related Applications

[0002] This application claims the benefit of both U.S. Provisional Application No. 62 / 520,755, filed on Jun. 16, 2017, entitled “TEST AND MEASUREMENT DEVICES, SYSTEMS AND METHODS ASSOCIATED WITH AUGMENTED REALITY” and U.S. Provisional Application No. 62 / 520,882, filed on Jun. 16, 2017, entitled “GESTURE BASED INTERFACE FOR INSTRUMENT CONTROL”, the contents of which are hereby incorporated by reference in their entirety. Technical Field

[0003] Embodiments of the disclosed technology generally relate to electrical test and measurement instruments and, in particular, to the use thereof to implement virtual reality and / or augmented reality. Background Art

[0004] Electrical test and measurement environments are dealing with increasingly complex signals, which are pushing the boundaries of the ability of test and measurement equipment to display corresponding waveforms in a manner intuitive to the end user. Similarly, the complexity of both the design and physical implementation of various devices under test (DUTs) is also increasing. This increased complexity of DUTs results in a corresponding increase in complexity when performing many tasks that may be required of the users of the test and measurement environment when analyzing the DUTs.

[0005] For example, a user of a test and measurement environment may need to identify a signal acquisition area on the design of a DUT, which may enable the acquisition of one or more signals of interest from the DUT. For example, this can be done by analyzing the design of the DUT (such as a schematic of the DUT) to identify areas or points in the design where one or more signals of interest can be acquired. Then, the user may need to physically locate a probe point associated with the signal acquisition area on the DUT itself to enable sampling of one or more signals of interest from the DUT.

[0006] For example, this can be achieved by applying one or more probes to a corresponding number of probe points that enable the probes to sample one or more signals of interest. Once the probes are applied, the user can view the signals on the display of a test and measurement instrument. However, due to the increasing complexity of the design of the DUT, the physical implementation of the DUT, and the signals generated by the DUT, each of these tasks continues to become increasingly difficult and time-consuming to perform.

[0007] In addition, a customer may have difficulty determining whether a particular contact is an acceptable location for connecting a ground clip or wiring (e.g., typically from a probe) because connecting the ground potential to an unacceptable location can damage the DUT and potentially become dangerous for the operator or the equipment.

[0008] Accordingly, there is still a need for improved devices, systems, and methods for the user experience in utilizing electrical test and measurement equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a block diagram illustrating an example of an electronic test and measurement environment having an augmented reality system incorporated therewith in accordance with certain implementations of the disclosed technology;

[0010] Figure 2A illustrates a first example of a physical field of view of a physical test environment in accordance with certain implementations of the disclosed technology;

[0011] Figure 2B illustrates a first example of an augmented image in accordance with certain implementations of the disclosed technology;

[0012] Figure 2C illustrates a first example of an augmented field of view in accordance with certain implementations of the disclosed technology;

[0013] Figure 2D illustrates a second example of an augmented image in accordance with certain implementations of the disclosed technology;

[0014] Figure 2E illustrates a second example of an augmented field of view in accordance with certain implementations of the disclosed technology;

[0015] Figure 2F illustrates a third example of an augmented field of view in accordance with certain implementations of the disclosed technology;

[0016] Figure 3A illustrates a second example of a physical field of view of a physical test environment in accordance with certain implementations of the disclosed technology;

[0017] Figure 3B illustrates a first example of an augmented field of view in accordance with certain implementations of the disclosed technology;

[0018] Figure 3C Illustrates a second example of an enhanced field of view according to certain implementations of the disclosed technology;

[0019] Figure 3D Illustrates a third example of an enhanced field of view according to certain implementations of the disclosed technology;

[0020] Figure 3E Illustrates a fourth example of an enhanced field of view according to certain implementations of the disclosed technology;

[0021] Figure 4 Is a flowchart illustrating an example of a processor control method for an electronic test and measurement system according to certain implementations of the disclosed technology;

[0022] Figure 5 Illustrates an example of a Smith chart according to various embodiments of the present disclosure;

[0023] Figure 6 Illustrates an example of data from a Smith chart according to various embodiments of the present disclosure, which is from Figure 5 The Smith chart illustrated;

[0024] Figure 7 Illustrates an example of an augmented reality (AR) environment according to various embodiments of the present disclosure;

[0025] Figure 8 Illustrates an example of virtual controls and waveform representations on the front panel of an oscilloscope according to various embodiments of the present disclosure, both of which are superimposed on a physical environment including a DUT;

[0026] Figure 9 Illustrates an example of a waveform representation according to various embodiments of the present disclosure, the waveform representation having visual cues superimposed on a physical environment to enable a user to more easily complete complex measurements;

[0027] Figure 10 Illustrates a first example of a design document according to various embodiments of the present disclosure, the design document being superimposed next to an associated DUT to enable a user to refer to a CAD design while viewing the physical DUT;

[0028] Figure 11 Illustrates a second example of a design document according to various embodiments of the present disclosure, the design document being superimposed next to an associated DUT to enable a user to refer to a CAD design while viewing the physical DUT;

[0029] Figure 12Illustrates examples of virtual overlays for guiding a user (e.g., repeating a past test cycle) during a test cycle while viewing a physical DUT, according to various embodiments of the present disclosure;

[0030] Figure 13 Illustrates examples of a virtual oscilloscope and a virtual infinite screen displayed in an AR environment, according to various embodiments of the present disclosure;

[0031] Figure 14 Illustrates examples of virtual depictions of temperature readings superimposed on a DUT, according to various embodiments of the present disclosure;

[0032] Figures 15A to 1 5E Illustrates attempts to show jitter currently through various eye diagrams and various bathtub diagrams;

[0033] Figures 16A to 16B Illustrates an example jitter map in an AR environment, according to various embodiments of the present disclosure;

[0034] Figure 17 Illustrates examples of aggregations of outputs from multiple test and measurement devices superimposed on a physical environment, according to various embodiments of the present disclosure;

[0035] Figure 18 Illustrates examples of virtual waveforms superimposed on a physical DUT, according to various embodiments of the present disclosure;

[0036] Figures 19A to 19C Illustrates examples of remotely viewing waveforms, according to various embodiments of the present disclosure;

[0037] Figure 20 Illustrates examples of virtual overlays of RF readings generated by probing a DUT and simultaneously viewing the physical DUT, according to various embodiments of the present disclosure;

[0038] Figures 21A to 21F Illustrates examples of manipulation (e.g., via gestures) of virtual data presented in an AR environment, according to various embodiments disclosed herein;

[0039] Figure 22 Illustrates examples of virtual three-dimensional objects representing analog / digital / RF waveforms that provide an enhanced immersive view, according to various embodiments of the present disclosure;

[0040] Figure 23 Is a schematic diagram of a system for gesture-based interaction with test and measurement instruments. Detailed Description

[0041] Embodiments of the disclosed technology generally relate to augmented reality systems that may be configured to be combined with an electrical test and measurement environment to enable a user to experience visual immersion within the electrical test and measurement environment. As described in more detail below, visual immersion of the user within the test and measurement environment can enable more efficient transitions between tasks that the user is performing.

[0042] Additionally, the system can present information and guidance to the user that was not possible at all prior to the augmented reality embodiments described herein. The user's visual immersion within the test and measurement environment can also enable the user to interact with signals, waveforms, and various other types of information and data in new and unique ways.

[0043] Such visual immersion can be achieved by overlaying virtual and / or augmented content on an image or video feed of the physical environment. In some configurations, an image or video feed of the physical environment can be captured by one or more cameras within the physical environment, such as those located on a head-mounted display (HMD), a tablet, etc. In other configurations, the virtual and / or augmented reality system can be configured to overlay virtual and / or augmented content on a transparent display that enables the user to directly view the physical environment with virtual / augmented images integrated with the physical environment.

[0044] In either configuration, the position of the virtual / augmented content within the physical environment can be determined, for example, based on one or more fiducials or other physical markers placed within the physical environment. Such fiducials can include aspects of the DUT itself (e.g., ball grid array, probe points, etc.), probes or aspects thereof, pointers within the physical environment (e.g., hand, stylus, etc.), or any other suitable artifact within the physical environment that can be used to identify the location at which the virtual / augmented content is placed or to initiate the generation of the virtual / augmented content.

[0045] It should be appreciated that the virtual / augmented content can also be placed according to a predetermined positioning within the display (e.g., determined by pixel coordinates) or the environment. For example, such a predetermined positioning can be automatically selected or set / changed by the user. It should be appreciated that such features are not limited by the mechanism used to determine the position of the virtual content. It should also be appreciated that any fiducial-based or fiducial-less virtual and / or augmented reality mechanism, or any combination thereof, can be used to determine when and / or where to display the various virtual / augmented content.

[0046] In various configurations, the orientation (e.g., perspective) of virtual / augmented content can be updated to reflect the movement of the user or the augmented reality system within the physical environment. For example, a user can view a waveform or its measurements in three-dimensional space. In such an example, the augmented reality system can enable the user to move in three-dimensional space to view the waveform or its measurements from different perspectives.

[0047] In alternative or additional configurations, the user can manipulate the perspective of the virtual / augmented content as if the user were holding a three-dimensional measurement. In such a configuration, the movement of the user's hand can be tracked (e.g., via a camera, infrared, laser, or any other suitable mechanism for tracking hand movement / gestures), and the virtual / augmented content can be updated to reflect those movements.

[0048] In certain embodiments, the AR environment can be configured with a ground warning system to visually highlight one or more areas that are not acceptable for grounding via a ground clip or wiring short; alternatively or additionally, the system can visually highlight contacts that are acceptable for grounding. This will provide visual guidance to advantageously allow the user to connect the probe ground to the exact location and eliminate any guesswork or user error.

[0049] Figure 1 is a block diagram illustrating an example of an electronic test and measurement environment 100 with an augmented reality system incorporated therewith according to certain implementations of the disclosed technology. In this example, the environment 100 includes test and measurement equipment 108, such as an oscilloscope, which is coupled to a probe 106 configured to interact with a device under test (DUT) 104. The DUT 104 can forward a waveform to the test and measurement equipment 108 via the probe 106 for processing (e.g., digitization, triggering, measurement, etc.) to generate waveform data (e.g., a digital representation of the waveform, waveform measurements, etc.) from the waveform.

[0050] The waveform data can then be forwarded to a computing device 110, which can be configured to format the waveform data for presentation to a user via a display device 112 such as, for example, a head-mounted display, a tablet device, an image display screen, or a mobile phone display. It should be appreciated that the data can alternatively be presented, for example, via a tablet with a rear camera for capturing an image of the physical environment on which augmented content can be superimposed. Any device capable of presenting the virtual / augmented content described herein superimposed on the physical environment is considered within the scope of the present disclosure. Thus, any reference herein to a head-mounted display can equivalently refer to any other suitable display device. Additionally, in certain embodiments, the display device 112 is at least partially transparent.

[0051] In some cases, the test and measurement environment 100 can include one or more data stores 114. The data store(s) 114 can be separate from the other depicted components (e.g., on one or more servers or other computing devices), or can be included within one or more of the depicted components. In some cases, the data store(s) 114 can provide information about the DUT 104 (e.g., design schematic, probe points, reference waveforms, etc.) or any other applicable information to the computing device 110 for presentation to a user via the head-mounted display 112. As such, the user can be enabled to view enhanced content while viewing the DUT 104 and / or waveform data associated with the DUT 104.

[0052] Although shown as multiple distinct components, it should be appreciated that any or all of the components in the test and measurement environment 100 can be integrated into fewer components (e.g., the computing device 110 and the head-mounted display device 112 can be combined into a single component), or can be expanded into more components. Although a single DUT 104 and test and measurement device 108 are depicted as single components, it should be appreciated that multiple DUTs and / or multiple test and measurement devices can be included within the test and measurement environment 100 without departing from the scope of the present disclosure.

[0053] In certain embodiments, the head-mounted display device 112 can have one or more cameras 116 integrated therewith. These cameras 116 can capture images from the physical environment. For example, images of the physical environment can be processed (e.g., by the head-mounted display device 112 or the computing device 110) to enable identification of location markers that can be used to identify an overlay location for placing enhanced content within the physical field of view of the physical environment. The cameras 116 can also include cameras integrated with the probe 106 and / or the DUT 104, which can be used as a source of enhanced content (e.g., Figure 2B enhanced image 220).

[0054] As used herein, enhanced content can include anything that is outside of the physical field of view but is processed to enhance the physical field of view. Such enhanced content can include, for example, a video feed from a camera integrated with the probe 106, a schematic of the DUT, waveform data captured by the probe 106, etc. Presenting such enhanced content in an augmented reality (AR) environment can provide certain advantages. For example, waveform data depicted in an augmented reality (AR) environment can advantageously allow a user to examine and interact with the content in three-dimensional (3D) form in the three-dimensional physical space in front of them. For example, the AR environment can be configured to allow the user to see the content in 3D form, rotate the content, walk around the content, zoom in and out of the content, and compress and expand the content. Controls coupled to the head-mounted display device 112 can also allow the user to interact with the waveform data (and / or waveform data sampled in real time by the oscilloscope 108) stored on the computing device 110 and / or the oscilloscope 108 to change the view of such waveform data.

[0055] In some embodiments, the data store 114 can be configured to store enhancement settings for dynamically enhancing the physical test environment. Such enhancement settings can include, but are not limited to, the position, orientation, or a combination thereof of the enhanced image within the display device 112. The computing device 110 can create an enhanced image based on the enhancement settings and, for example, input feeds received from the physical test environment by the probe 106 or the camera 116. The computing device 110 can then output the enhanced image to be superimposed on the physical test environment to enhance the user's view of the physical test environment, for example, by causing the display device 112 to visually present the enhanced image superimposed on the physical test environment to the user. It should be appreciated that where the display is at least partially translucent, the enhanced image can be displayed to align with aspects of the physical test environment viewable through the translucent display, thereby causing the enhanced image to be superimposed on the physical environment. In such cases, an image of the physical test environment can be captured and used to determine aspects of the physical environment for placement of the enhanced content within the enhanced image.

[0056] Generally, AR methods make content virtually tangible and, as such, open up new exploration modes that were not previously available on conventional fixed displays. For example, AR methods can allow multiple users to view within a shared, real-world physical space by employing multiple head-mounted displays coupled via a network (e.g., wired, wireless, or any combination thereof) to one or more computing devices.

[0057] Waveform data generated by oscilloscope 108 and / or other test and measurement equipment can be used as augmented content in an AR environment. For example, controls can allow a user to virtually place a virtual representation of a waveform at a physical location on a circuit board or at the physical location where a corresponding signal physically resides on DUT 104. The virtual representation of the waveform can be virtually placed on the user's desk for further analysis, on a conference table for simultaneous viewing by multiple users, etc.

[0058] By using AR methods to display static waveform data and / or stream dynamic / real-time waveform data, a user can utilize the head-mounted display device 112 to view the waveform in true 3D form as if the waveform data were a physical object within the physical environment surrounding the user. This represents an unprecedented degree of freedom in the exploration of waveform data, thereby allowing one or more users to simultaneously view the virtual waveform in physical space from multiple viewpoints (such as from below, behind, on top of, and / or inside the waveform). For example, this can allow a user to debug and / or characterize complex devices or circuits. This can also allow a user to directly view the physical area being probed and view the virtual waveform floating above the probe capturing the waveform.

[0059] Figure 2A A first example of the physical field of view of a physical test environment 200 according to certain implementations of the disclosed technology is illustrated. In this example, DUT 210 is visible within the physical test environment 200. Camera 214 can be integrated with the probe / probe tip 212, and the image or video from this camera can be streamed into the AR environment so that the user can effectively have a heads-up display (HUD) with a magnified view of the area being probed while still viewing DUT 210, the probe position, and / or the test and measurement equipment. This can enable the user to more accurately place the probe tip at a desired probing point, which can result in a better electrical connection between the probe tip and the probing point.

[0060] Figure 2B A first example of an augmented image 220 according to certain implementations of the disclosed technology is illustrated. In this example, the augmented image 220 is, for example, a magnified view of the area being probed where the user is currently positioning a probe on DUT 210. In some embodiments, the augmented image can include the generated waveform associated with DUT 210 in the physical test environment 200.

[0061] Figure 2CIllustrates a first example of an enhanced field of view 200 according to certain implementations of the disclosed technology. In this example, a user is able to use a display device such as a head-mounted display device to simultaneously view a physical test environment including a DUT 210, a probe 212, and an enhanced image 220, e.g., a magnified view of the probe or the probing area of the DUT 210. The position, orientation, etc. of the enhanced image can be selected and / or changed by the user using enhancement settings.

[0062] Figure 2D Illustrates a second example of an enhanced image 230 according to certain implementations of the disclosed technology. In this example, the enhanced image 230 is a waveform associated with a probe 212 or a probing location on the DUT 210.

[0063] Figure 2E Illustrates a second example of an enhanced field of view 200 according to certain implementations of the disclosed technology. In this example, a user is able to use a display device to simultaneously view a physical test environment including the DUT 210 and an enhanced image 230, e.g., a waveform associated with the probe or the probing area of the DUT 210. The position, orientation, etc. of the waveform can be selected and / or changed by the user using enhancement settings.

[0064] Figure 2F Illustrates a third example of an enhanced field of view 200 according to certain implementations of the disclosed technology. In this example, in a single immersive view, both a magnified view of the probing area 220 and a waveform 230 are superimposed on a physical environment including the DUT210. Such an immersive view can advantageously allow the user to improve both concentration and efficiency. This implementation also has the advantage that a bench microscope (which is becoming increasingly necessary when working with microelectronic devices) may no longer be necessary, which can save costs and also result in improved processing, both of which provide benefits that can be directly passed on to the customer.

[0065] In certain embodiments, the system can be configured to selectively guide a user to probe specific points on a DUT. This can be particularly beneficial for complex circuit boards and can increase productivity and reduce errors in testing, while also enabling test repeatability. In such embodiments, a virtual overlay can be generated from, e.g., user inputs such as CAD files and test point selection data, and the virtual guidance can be made to overlay the physical DUT to allow the user to probe exact locations and eliminate guesswork or user error in locating exact test points for a particular test. The user can upload an appropriate data file in software and then select a single test point, or be guided through a sequence of test points by visual indications such as colored dots or symbols of a virtual overlay as discussed in the examples below.

[0066] In some cases, an engineer or technician may want to record their work. When their work is carried out in an AR environment configured as described herein, they can see the progress and results related to the physical environment around them, but now they may expect a way to record these visual scenes so that the results can be shared with others or saved for future reference.

[0067] In such a case, the AR environment described herein can be configured to perform a virtual AR screen dump utility or AR screen recording, which can record a copy of whatever the user sees (both virtual and physical) within the AR environment at a certain point in time or over a period of time. In embodiments utilizing a transparent display, the capture of the physical environment can include appropriately placed virtual content so that the AR environment can be correctly captured within the document.

[0068] Figure 3A A second example of the physical field of view of a physical test environment 300 according to certain implementations of the disclosed technology is illustrated. In this example, the DUT 350 is visible within the physical test environment 300. The DUT 350 has various potential probe points and connections therebetween, for example, including 352, 354, and 356.

[0069] Figure 3B A first example of an enhanced field of view 300 according to certain implementations of the disclosed technology is illustrated. Here, the enhanced image includes a virtual user interface having multiple parts such as a first information portion 330 and a second information portion 320 (such as a virtual control panel having multiple virtual controls). The first information portion 330 includes a selection component 310, which can provide an indication of an area to be tested or an area to be avoided. The first information portion 330 may also include a waveform portion 340, which is configured to display, for example, a waveform associated with the area being tested. The second information portion 320 may include a number of actions that the user can select.

[0070] In this example, the virtual overlay also includes probe points on the physical environment, for example, to help prevent the user from incorrectly grounding the test and measurement system. In this example, the "+" dots indicate acceptable ground contacts used by the probe ground clip or wiring. In this example, the virtual overlay also includes dots to indicate valid and / or applicable points for testing. It should be appreciated that other symbols, icons, colors, etc. can be used to indicate such contact points.

[0071] Such a ground warning system can be implemented by storing the location of the contacts on the DUT 350 (e.g., coordinate location), physical markers, or other suitable location information. The location information can then be associated with an indicator of whether the location is an acceptable ground. Based on the location information and the association, an overlay image 300 can be generated and overlaid on the DUT such that an operator can easily identify those locations that are acceptable and / or unacceptable for ground connections. For example, such an image or its overlay location can be updated in real time even when the user moves relative to the DUT 350 within the physical environment to keep the indicator in place.

[0072] Figure 3C FIG. illustrates a second example of an enhanced field of view 300 according to certain implementations of the disclosed techniques. In this example, the user interacts with the selection component 310 to select a particular probe point. The virtual tag 360 provides a visual indication of the physical location of the probe point on the DUT 350, e.g., to allow the user to easily locate the probe point and establish a connection with the probe point by means of the probe tip. Once the user has established a connection with the probe point, the waveform portion 340 can visually present the waveform corresponding to the probe point.

[0073] Figure 3D FIG. illustrates a third example of an enhanced field of view 300 according to certain implementations of the disclosed techniques. In this example, the user interacts with the selection component 310 to select an additional probe point. The virtual tag 360 again provides a visual indication of the physical location of the probe point on the DUT 350, e.g., to allow the user to easily locate the probe point and establish a connection with the probe point by means of the probe tip. Once the user has established a connection with the probe point, the waveform portion 340 can be dynamically changed / updated to visually present the waveform corresponding to the newly selected probe point.

[0074] Certain embodiments can include placing a virtual marker in the enhanced image in response to the user causing a measurement probe to physically contact a particular portion within the physical test environment. Figure 3E FIG. illustrates a fourth example of an enhanced field of view 300 according to certain implementations of the disclosed techniques. In this example, the virtual marker includes a box 399 that is positioned around the probe point where, for example, a connection with a probe is maintained. That is, the virtual marker corresponds to the identified location of the particular portion. This can advantageously allow the user to easily visually confirm that the probe remains connected to a particular point on the DUT 350 while the user is performing one or more other activities that may be at a physical location separate from the physical test environment.

[0075] Figure 4 is a diagram illustrating, according to certain implementations of the disclosed techniques, for an electronic test measurement system (such as byFigure 1 Flowchart of an example of a processor-controlled method 400 of an illustrated electronic test and measurement system).

[0076] At 402, enhancement settings are stored, for example, by one or more data stores. The enhancement settings can be used by the system for dynamically enhancing a physical test environment. The enhancement settings can indicate the positioning, location, orientation, or a combination thereof of enhanced content to be visually presented to a user, for example, by a display device. In some embodiments, the enhancement settings can be configurable to allow a user to select a positioning within a display device for an enhanced image. In some cases, any or all of these enhancement settings can be dynamically edited by the user.

[0077] At 404, an input feed is received from the physical test environment, for example, by a computing device. The input feed can include, for example, a data feed captured by a probe, a video feed captured by a camera attached to the probe, or a representation of the physical environment (e.g., a stored representation). In some embodiments, a camera can be integrated into the display device.

[0078] At 406, an enhanced image is created, for example, by a computing device. The enhanced image can include a virtual overlay of virtual or enhanced content on the physical environment to enhance a user's view of the physical test environment. The enhanced image can be created based on one or both of the enhancement settings and the input feed.

[0079] At 408, the enhanced image created at 406 is output, for example, by a computing device. For example, the computing device can output the enhanced image to any one of a number of suitable display devices. Alternatively or additionally, the computing device can output the enhanced image to one or more storage devices, for example, for later use by the user or other users.

[0080] At 410, the enhanced image is visually presented to a user by a display device such as a head-mounted display, a tablet device, an image display screen, or a display of a mobile phone. The display device can be at least partially transparent. In some implementations, the enhanced image can be visually presented to multiple users, for example, by way of multiple display devices. For example, a first user can interact with a head-mounted display device while a second user can view the interaction of the first user via a conventional desktop computer or a tablet device.

[0081] Some implementations can include a measurement probe that is user-controllable to interact with the physical test environment, where a camera is coupled to the measurement probe. In such embodiments, the computing device can provide instructions to the user and further change the instructions in response to input received from the user by way of the measurement probe.

[0082] Some embodiments may include using the systems or techniques described herein with a time domain reflectometer (TDR). For example, the system may enable a user to use a display device to visualize TDR traces on a circuit board and an enhanced image including the computed impedance or any other suitable measurements associated with the TDR traces.

[0083] Regarding Figure 5 and Figure 6 , the Smith chart is a common way to display the relationship of impedance versus frequency for RF measurements and analysis (see Figure 5 ). Typical usage includes making impedance measurements in the complex plane for a given DUT. Without changing any test parameters, subsequent measurement results show the same Smith chart impedance trace for each sweep analysis. However, a common test parameter adjustment is the RF power level, which changes the DUT impedance and the resulting Smith chart. Typically, RF engineers analyze this data as a set of separate data sets, one set for each step in the power level of the DUT corresponding to the impedance versus frequency analysis.

[0084] However, if these same Smith chart measurements are stacked together (e.g., by a test and measurement device) into the Z-axis, a 3D representation can be generated that can be viewed / manipulated in an AR environment, such as illustrated by Figure 1 . This provides a first visual tool of this kind for RF engineers to help visualize how impedance versus frequency changes with the power level. Then, the 3D representation can be rotated, translated / scaled, separated (e.g., by expanding one or more axes), and explored in 3D space with RF markers. As a simple example, if the resulting 3D representation is rotated 90 degrees about its axis, the curves between data sets can be easily identified, and power tuning analysis can be accelerated (see Figure 6 ). Further, observable phenomena such as the relationship between data sets in three-dimensional space can be more easily identified — a difficult task otherwise if the user is limited to analyzing individual data sets.

[0085] Modern oscilloscopes limit users to viewing their data on a fixed two-dimensional (2D) screen. Typically, this also limits the presented data to 2D form — that is, there are no easily accessible three-dimensional (3D) viewing options. For example, histograms attempt to convey a kind of “third dimension” in waveform data (e.g., the number of samples or the stacking of data points within a given histogram “bin”), but are limited by the fixed 2D screen. Existing MATLAB-based solutions that provide 3D representations of MATLAB-generated waveform data are similarly limited to a fixed 2D display. Augmented reality (AR) methods open the door for unique and novel 3D displays and interaction with content in real-world settings.

[0086] The AR environment can be configured so that users can fill physical locations with virtual content. This expands the display technology from the fixed 2D displays on the instrument and creates a highly customizable AR virtual environment (see Figure 7 ). This allows users to maximize the screen space of a physical oscilloscope or the screen space of other test and measurement equipment by moving multiple waveforms or GUIs (Graphical User Interfaces) out of the physical screen and onto the table or into the air around the user, instrument, DUT, etc.

[0087] With AR, all the traditional limitations of a fixed 2D display are removed, such as a fixed size and a fixed position, as well as a limited size, because the virtual environment is not necessarily restricted by physical space. In other words, the room or space around the user now becomes the display for the instrument output, whether it is waveforms or measurement data. AR can be viewed via modern display technologies, whether it is a mobile device (such as a phone or a tablet), a computer monitor, a projector, or through a head-mounted display (such as Oculus Rift, HTC Vive, Microsoft Hololens, etc.).

[0088] In some embodiments, the AR environment can also be configured to virtualize the controls of the instrument into virtual representations. Figure 8 A heads-up display in AR of the oscilloscope front panel or a subset thereof or the controls for a solution application (e.g., 80SJNB) is depicted so that the user can continue to view the DUT while also controlling the settings of the test and measurement instrument. This can include tracking (e.g., camera, infrared, laser, eye tracking, etc.) the user's movement in order to be able to select their input on the front panel displayed in AR or move to another area of the control panel where the virtual control panel is truncated due to any display area limitations. Like any overlay, the virtual front panel may be opaque or transparent to varying degrees. Whether from the table to the laboratory or from the office to the field, this feature opens the door to increased productivity and enhanced remote oscilloscope control. In such a configuration, the test and measurement equipment that is controlled by changing the settings of the virtualized controls can be configured to receive signals from the AR environment indicating the changes made to the virtual controls so that the test and measurement equipment can implement such changes.

[0089] In some cases, to obtain the desired measurement values, 3D graphs are generated and other complex operations are performed by the user for input. In many such cases, this is an operation that visually requires 3D input, which can complicate matters even if the customer is currently restricted to 2D input. In terms of complexity, consider visual triggering on an oscilloscope available from TEKTRONIX®. Visual triggering enables the user to achieve the following option: effectively create a trigger based on visual elements of a waveform specified by the user (e.g., specific points within the waveform). Instead of wasting time on complex trigger settings and other considerations to obtain the desired result, the user can simply point to the displayed element from which complex trigger settings can be derived. In an AR environment, this is already better suited to display 3D results. AR visual measurement setup features can speed up complex measurement setups (e.g., using Figure 9 the points identified within). Further, in such an AR environment, visual triggering can also be extended into the 3D realm.

[0090] In some cases, when physically evaluating or debugging a DUT, engineers often need to refer to background information of the DUT (e.g., schematic diagrams, block diagrams, etc.). In a conventional setup, this requires looking back and forth between the physical DUT and a computer monitor or hard copy (e.g., a CAD printout). In the AR environment described herein, the user can be immersed in an environment where the background information is superimposed on the DUT, thus eliminating the need to look back and forth (see Figure 10 ). Now, the background information is seen by the user as a 2D or 3D floating virtual display shown side by side with the physical DUT. The ability to change the color of the transparent overlay can eliminate visibility issues in cases of different background colors; optionally, an opaque background can also be used to completely avoid the problem (see Figure 11 ). As shown, other information can also be virtually superimposed to assist the user.

[0091] When testing electromagnetic interference (EMI) radiation and collecting test data during the debugging or verification phase of the design cycle, a common challenge is the repeatability of these tests. In some configurations, the AR environment described herein can be configured to track and record such tests for future reference. This can include the position of the probe relative to the DUT (e.g., using position markers identified within the DUT and / or motion sensors within the probe). When referring back to the recorded test at a later time (e.g., selected by the user via a user interface), the AR environment can be configured to generate a virtual path based on one or more previous tests. Then, the AR environment can be configured to superimpose the virtual path on top of the DUT based on the position markers. This can enable the AR environment to effectively guide the user in replicating the test.

[0092] Such visualization can assist the user, for example, in guiding the movement of a near-field probe as to where to move it in order to repeat a previous test more closely (see Figure 12 ). This allows the user to see something they couldn't see before - the previous test path (e.g., an EMI test path). The data collected therefrom can also be submitted as verification data so that standards / approval bodies can repeat or share more precise pass / fail criteria and data with companies attempting to certify a DUT through the standards / approval bodies. As shown, other information can also be virtually overlaid to assist the user. This can also be used in an environmental chamber, and the data collected can also be used for viewing during offline analysis (e.g., to compare two or more tests).

[0093] By sending waveform data into the AR environment, there is virtually no practical limit to the screen size of the oscilloscope. It can be regarded as an "extended view" or "virtual memory" or "infinite memory" view of the waveform data. That is, given a particular data set, the waveform presented on the screen doesn't have to stop here - it can continue and keep moving left in the same visual plane of the oscilloscope screen. Similarly, the incoming waveform data (earlier part of the record) can also be presented on the right side of the same visual plane (see Figure 13 ). This also opens the door to enhanced AR display options such as pan / zoom and MagniVu features from, for example, Tektronix®.

[0094] Although depicted as being displayed on a virtual oscilloscope, it should be appreciated that such waveform data can be presented without the need for a visual oscilloscope. To traverse different parts of the waveform, the user can perform any number of tasks, such as for example sliding a hand in one direction or the other to cause the waveform to move according to the direction of the slide; virtually grabbing a portion of the waveform and moving the waveform in either direction via the virtually grabbed portion, rotating in one direction or the other, stepping in one direction or the other, or performing any number of additional interactions.

[0095] In some embodiments, the AR environment can be configured to generate temperature data content (collected from sensors) and overlay this content in real time on the associated DUT for immediate viewing (see Figure 14 ). Further, this data can be collected for offline analysis, and then the offline analysis can be used to provide a benchmark for future tests. This enables regression testing of the DUT when design changes and other modifications are made. Other information can also be virtually overlaid to assist the user. This technology can also be used in environmental chambers and a variety of other test locations.

[0096] In some embodiments, to gain insight into additional jitter measurements, a user can view 3D representations, such as bathtub plots or eye funnel plots. Currently, these methods are limited to a fixed 2D display. In some cases, an AR environment can be configured to enable a user to visually "drill down" and further explore these visualizations in an interactive 3D environment. Thus, it is more convenient to use a cursor on a 3D bathtub plot by enabling the user to manipulate virtual objects of the bathtub plot by virtually moving a cursor within the physical space (e.g., via gestures, HMI, or other control inputs) and visually placing the cursor in a more exact location. Providing a true 3D representation via the AR environment ultimately provides a more intuitive rendering of the 3D data that the user needs to conduct their analysis. According to various embodiments of the present disclosure, Figures 15A to 15D illustrates current attempts to show jitter via various eye diagrams and various bathtub plots, and Figures 16A to 16B illustrates an example jitter plot in an AR environment.

[0097] Conventional oscilloscopes may also be limited in terms of the number of channels that can be displayed simultaneously. That is, even if additional channels are from other oscilloscopes connected together, currently it is not possible to display more than the typical 2 to 8 channels on a single display of a modern oscilloscope. To address this issue, an AR environment can be configured to aggregate channels from multiple oscilloscopes into a single virtual view. Since this can be a pure display aggregation, it may not be necessary to synchronize multiple oscilloscopes to accomplish this operation. In other configurations, multiple oscilloscopes can be synchronized by any conventional synchronization method to time-align signals from multiple oscilloscopes. In an embodiment, the AR environment can be configured to provide additional background information (e.g., the location of the DUT that generated the waveform).

[0098] An example application is the embedded automotive market, where there are many sensors and many electrical test points (see Figure 17 ). As depicted, each waveform is located near the DUT or probe point that generated that waveform. This can be achieved by associating each oscilloscope channel, waveform, or DUT with a physical marker or other location information to enable identification of the appropriate location to superimpose the waveform within the physical environment. In some cases, such as in the depicted engine compartment, the proximity of the waveforms being displayed may make it impossible for all of these waveforms to fit within a suitable viewing size without overlapping. In such an embodiment, the AR environment can be configured to generate a dashed line from the identified location of the physical marker to the associated waveform plot.

[0099] This method utilizes AR to allow customers to leverage their existing installed base or purchase multiple next-generation oscilloscopes and synthesize a higher total number of channels for their applications. The key is to re-route the outputs of multiple oscilloscopes (waveforms) together into the AR environment and then be uniformly viewable by the user (see Figure 17 ). Some channel differentiation schemes (e.g., color differentiation, pattern differentiation, etc.) can be implemented to avoid displaying multiple (but different) channels in an indistinguishable manner. For example, different available colors can be assigned to each oscilloscope channel to avoid having multiple channels with the same displayed color, thus eliminating confusion in the AR environment.

[0100] In some embodiments, the AR environment can be configured to virtually pair the waveform data with the probe (e.g., via a physical marker on the probe). For example, the AR environment can be configured to overlay a virtual representation of the waveform on the circuit board under test or device close to the physical location where the signal physically resides, as illustrated by Figure 18 In an example, the AR environment can be configured to enable the user to move the waveform from the DUT to, for example, the user's desk for further analysis or to a conference table for multiple users to observe simultaneously.

[0101] In such an AR environment, the user can utilize a head-mounted display to view the waveform in true 3D as if it were a real physical object in front of them. This represents an unprecedented degree of freedom in exploring waveform data, allowing one or more users to simultaneously view the virtual waveform from below, behind, on top of, or even from inside the waveform itself in physical space. Imagine debugging or characterizing a complex device or circuit and being able to directly see the physical area being probed and see the virtual waveform floating above the probe in the user's hand.

[0102] Regarding Figures 19A to 19C , the AR environment can be configured to enable remote viewing of the waveform data generated by test and measurement instruments. As used in the context herein, "remote viewing" of waveform data can mean being removed from the physical location of the DUT and / or test and measurement instrument, including whether the waveform is offline as previously saved waveform data (such as a reference waveform or a saved waveform file) or whether the waveform data is real-time waveform data from a real-time feed of the real waveform.

[0103] For example, the AR environment can be configured such that a user can place a virtual representation on the user's desk for further analysis, or on a conference table for simultaneous viewing by multiple users for collaboration (e.g., via multiple head-mounted displays, where each user can view the waveform data from the perspective of the user's position relative to the waveform data). Graphical elements such as gridlines and other readouts would be optional and user-selectable (e.g., via a menu in the AR environment). This remote approach can also be combined with real-time local waveforms to complete the comparison function between the saved / reference waveform and the real-time local waveform.

[0104] By using the AR method to display (static) and / or stream (dynamic / real-time) waveform data, a user can view the waveform in true 3D form using a head-mounted display device as if it were a real physical object in front of them in physical space. This represents an unprecedented degree of freedom in exploring waveform data, allowing one or more users to simultaneously see the virtual waveform from below, behind, on top of, or even from inside the waveform itself in physical space.

[0105] In some cases, when testing a DUT operating in the radio frequency (RF) domain, a designer may resort to the old and cumbersome process of tabulating readings in a spreadsheet. This typically involves entering physical location information in a meaningful way, such as by component number, network reference, etc. As such, the designer may constantly switch between taking readings and recording them in the spreadsheet. To address this issue, the AR environment can be configured such that RF data acquisition tests can be tracked, associated with physical locations on the DUT, and plotted in real-time for immediate viewing or for future offline reference.

[0106] When a designer or tester begins testing and guides an RF probe through the DUT, the AR environment can be configured to track the position information of the RF probe relative to the DUT (e.g., via physical markers or other suitable position tracking information), and record a 3D map of the RF spectrum readings. The AR environment can then be configured to plot the RF spectrum readings as an overlay on the DUT (see Figure 20 ). By overlaying the RF spectrum readings on the DUT, the spectrum readings can be presented in situations that would not be possible in a conventional test and measurement environment.

[0107] Compared with the cumbersome manual spreadsheet entry, this highly automated application fills a technology gap and allows users to see what they couldn't see before - a true physical location map of RF readings emitted near the DUT. Further, this data can be collected for offline analysis, and then the offline analysis can be used to provide a benchmark for future tests. This enables regression testing of the DUT when design changes and other modifications are made. Other information can also be virtually overlaid to assist the user. This technology can also be used in environmental chambers and various other test locations.

[0108] One aspect of the AR environment is the ability to manipulate the display of 2D or 3D virtual content. Thus, the AR environment can be configured to enable virtual manipulation of waveform data using a human-machine interface (HMI, such as a remote HMI, or other controller input) or directly with the hand based on heuristics or gestures. This includes, but is not limited to, actions such as zooming, rotating, rolling, yawing, pitching, and virtual moving / placing of overlays in the virtual environment (see Figures 21A to 21D ). Thus, the display capabilities of the AR environment can far exceed what can be observed from waveform data when restricted to the fixed 2D displays of conventional test and measurement instruments (see Figures 21E to 21F ). Graphical elements such as gridlines and other readouts can be optional in the AR environment and are selectable by the user via the menu of the AR environment.

[0109] By using an AR display (referring to a static display in this case) or streaming (referring to a dynamic / real-time display) of waveform data, users can use a head-mounted display to view waveform data in true 3D form, as if the waveform data were a real three-dimensional physical object in front of them. This represents an unprecedented degree of freedom in exploring waveform data, allowing one or more users to simultaneously see a virtual representation of the waveform data from below, behind, on top of, and even inside the waveform itself in physical space.

[0110] Oscilloscopes limited by fixed 2D displays have tried to convey extra-dimensional data in the past. For example, RF spectrum analyzers and mixed-domain (supporting RF functions) oscilloscopes can include a view (e.g., the spectrum view from Tektronix®) where RF traces can be stacked over time - however, even in the spectrum view, the display, and thus the data, is still limited to two dimensions. Introducing analog, digital, and RF signals into 3D space provides new opportunities for increased insight into testing and measurement.

[0111] The AR environment can be configured to stack time-varying information of the waveform into the third dimension (see Figure 22), which can be viewed to enhance the view of the signal data. This can include qualities such as phase, jitter, real points relative to imaginary points, and constellation diagram data. Further, the AR environment can be configured to generate 3D histograms or 3D X-Y plots, and the 3D histograms or 3D X-Y plots can be mined via interaction (e.g., via gestures) with these 3D histograms or 3D X-Y plots in the 3D physical space for additional insights. Additionally, the AR environment can be configured to generate a topographical map of the signal and enable exploration of the 3D persistence of the waveform.

[0112] In further embodiments, the AR environment can be configured to perform many other tasks. For example, the screen space of a physical oscilloscope can be maximized by moving multiple waveforms and / or graphical user interface (GUI) windows out of the physical oscilloscope screen and into the space outside the oscilloscope. Another example is a true 3D display of a 3D histogram where the waveform grows and shifts in all three dimensions over time. Another example is a measurement category based on true 3D waveform data parsed into individual samples or groups of samples that are only accessible in the virtual 3D space.

[0113] To enable interaction with test and measurement devices, a visual interface is also disclosed herein that can monitor gesture-based two-dimensional or three-dimensional input for test and measurement devices. Such input can be captured by any number of devices or combinations of devices, such as one or more cameras, infrared sensors, laser sensors, motion sensors including contact motion sensors (e.g., gloves with motion sensors) or non-contact motion sensors, etc. It should be appreciated that the embodiments disclosed herein are not limited by the manner in which gestures are captured and envision any gesture capture mechanism. By enabling users of test and measurement instruments to interact with the test and measurement instruments while being away from them, the present disclosure enables the ability to control test and measurement instruments while being physically isolated from the machine. In some cases, this can enable control of test and measurement instruments in potentially hazardous scenarios (such as high-voltage scenarios) or in scenarios where the user's interaction with the test and measurement instrument may affect the test results. In some cases, the present disclosure can also enable users to control test and measurement instruments through a transparent isolation, such as a window in an isolation chamber or a thermal chamber. Thus, users can control test and measurement instruments in environments that may be inhospitable or dangerous to humans. In other cases, aspects of the present disclosure can be implemented in an augmented reality environment such as described herein to enable users to control test and measurement instruments while immersed in an augmented reality experience.

[0114] By using gestures, there is no limitation on the size of the device used to support the interface, and in addition to the gesture interpretation module, any device used to support the interface will only need to implement a gesture capture mechanism, such as those illustrative examples described above. The gesture interpretation module can be configured to detect and interpret gestures from the data provided by the gesture capture mechanism and associate the gestures with command signals to be provided to the test and measurement instrument to implement the commands associated with the gestures. In some cases, this may involve fairly simple edge-detection-based gesture recognition and access to the test and measurement instrument interface. In some cases, the gesture capture mechanism and the gesture interpretation module can be integrated with the test and measurement instrument, and in other cases, the gesture capture mechanism and / or the gesture interpretation module can be external to or remote from the test and measurement instrument.

[0115] Unique gestures or gesture sequences can be used to access and manipulate various subsystems. For example, a start gesture can be used to identify a subsystem, and then other gestures can be used to perform tasks within that subsystem. Thus, tasks in different subsystems can be associated with the same gesture; however, because the situations (e.g., subsystems) are different, the same gesture can implement different tasks. Additional details are described in more detail below.

[0116] By using a gesture-based interface, a test and measurement instrument can be controlled without a physical or electrical connection. This can open up possibilities for more sensitive measurements and for controlling instruments in hazardous environments. Additionally, by using a new form of interface, it also expands the possible controls beyond the currently available controls.

[0117] By using a gesture-based interface, limitations on physical access to the test and measurement instrument or remote input device are avoided. By allowing the user to interactively control the instrument, the need for prior knowledge of programmatically remotely controlling the instrument is reduced. Additionally, in embodiments using a visual method, interference from noise within the test environment is mitigated.

[0118] In some configurations, the present disclosure combines physical gestures, a gesture capture mechanism, a gesture interpretation module, and a test and measurement instrument including a command interface to provide a contactless method for controlling the test and measurement instrument. Figure 23 Illustrative components for such a system are identified.

[0119] The system provided by the present disclosure uses gestures observed by a gesture capture mechanism, which is connected to a gesture interpretation module to interpret the gestures as specific commands, and then the commands can be communicated to a test and measurement instrument via the command interface of the test and measurement instrument. In some embodiments, gesture interpretation can be achieved by a simple edge detection and tracking mechanism or more complex methods.

[0120] The gesture interpretation module will identify the gesture and convert it into a command step request. These command steps can be routed to a test and measurement device instrument through an internal or external interface. Then, the test and measurement instrument or any other component can provide feedback to the user that the gesture has been recognized. In an alternative embodiment, the gesture interpretation module can establish a graphical or alphanumeric representation of the command sequence to allow the user to confirm the command sequence, and then use another gesture to initiate the command and send it to the control interface of the test and measurement instrument.

[0121] Gestures can be implemented in many ways, for example, by a combination of linear or circular movements that can be tracked by the gesture capture mechanism. Gestures can be interpreted sequentially to provide a complete command to the test and measurement instrument. General gestures can include (but are not limited to) one-handed movements in a linear manner (horizontally, vertically, or diagonally), one-handed movements in a circular motion (clockwise, counterclockwise), multi-handed movements where multiple hands move relative to each other (separated or joined together vertically, horizontally, or diagonally), finger movements alone or in combination, movements of any other part of the body, or movements of inanimate objects associated with the gesture.

[0122] The command structure can be implemented as follows: First, use a unique initiation gesture to identify the set of commands to be accessed, and then use one of a set of gestures that indicate the next step in the command sequence. When establishing a command sequence, the command gestures can be repeated. The command sequence can be effectively recursive as needed to support nested commands, such as menu selections.

[0123] The following is a set of basic gestures from which the command set can consist of:

[0124] One-handed horizontal movement from left to right - where the hand is oriented perpendicular to the direction of movement for the simplest detection;

[0125] One-handed horizontal movement from right to left - where the hand is oriented perpendicular to the direction of movement for the simplest detection;

[0126] One-handed vertical movement downward - where the hand is oriented perpendicular to the direction of movement for the simplest detection;

[0127] One - hand vertical upward movement - where the hand is oriented perpendicular to the direction of movement for the simplest detection;

[0128] One - hand diagonal movement from upper right to lower left - where the hand is oriented perpendicular to the direction of movement for the simplest detection;

[0129] One - hand diagonal movement from lower right to upper left - where the hand is oriented perpendicular to the direction of movement for the simplest detection;

[0130] One - hand diagonal movement from upper left to lower right - where the hand is oriented perpendicular to the direction of movement for the simplest detection;

[0131] One - hand diagonal movement from lower left to upper right - where the hand is oriented perpendicular to the direction of movement for the simplest detection;

[0132] One - hand clockwise rotational movement;

[0133] One - hand counter - clockwise rotational movement;

[0134] Two - hand horizontal movement apart - where the hands are oriented perpendicular to the direction of movement for the simplest detection;

[0135] Two - hand vertical movement apart - where the hands are oriented perpendicular to the direction of movement for the simplest detection;

[0136] Two - hand diagonal movement apart towards upper left and lower right - where the hands are oriented perpendicular to the direction of movement for the simplest detection;

[0137] Two - hand diagonal movement apart towards lower left and upper right - where the hands are oriented perpendicular to the direction of movement for the simplest detection;

[0138] Two - hand horizontal movement towards each other - where the hands are oriented perpendicular to the direction of movement for the simplest detection;

[0139] Two - hand vertical movement towards each other - where the hands are oriented perpendicular to the direction of movement for the simplest detection;

[0140] Two - hand diagonal movement towards each other from upper left and lower right - where the hands are oriented perpendicular to the direction of movement for the simplest detection;

[0141] Two - hand diagonal movement towards each other from lower left and upper right - where the hands are oriented perpendicular to the direction of movement for the simplest detection.

[0142] There are also other possibilities since these are combined into a single gesture to form stylized alphabetic characters.

[0143] Here are some examples of gesture sequences and what they might represent for a test rig. It is not intended to be exhaustive or complete.

[0144] Enter horizontal timebase control - one hand moves from left to right (or any other suitable gesture)

[0145] Timebase control activated:

[0146] Two hands move horizontally apart - expand timebase, less time per division;

[0147] Two hands move horizontally together - shorten timebase, more time per division;

[0148] One hand moves from left to right - scroll display window to a later position in the trace display;

[0149] One hand moves from right to left - scroll display window to an earlier position in the trace display;

[0150] One hand moves vertically up - exit timebase control mode.

[0151] Enter vertical sensitivity for the active channel - one hand moves vertically up (or any other suitable gesture)

[0152] Vertical channel control activated:

[0153] Two hands move vertically apart - increase channel sensitivity, less voltage per division;

[0154] Two hands move vertically together - decrease channel sensitivity, more voltage per division;

[0155] One hand moves vertically up - move the ground reference up on the display;

[0156] One hand moves vertically down - move the ground reference down on the display;

[0157] One hand moves horizontally from left to right - exit vertical channel control mode.

[0158] Enter trigger mode control - one hand moves diagonally from bottom left to top right (or any other suitable gesture)

[0159] Trigger mode control activated:

[0160] One hand moves vertically up - increase the trigger level;

[0161] One hand moves vertically down - decrease the trigger level;

[0162] One hand moves horizontally from left to right - move the trigger point further back in the trace;

[0163] Moving a single hand horizontally from right to left - Moving the trigger point forward in the trace;

[0164] Moving a single hand diagonally from upper left to lower right - Switching the trigger mode;

[0165] Moving a single hand clockwise - Entering the channel selection mode. In this mode, the selected channel will automatically deselect the previously triggered channel. Operate in this mode using the following rules:

[0166] Moving a single hand diagonally from upper right to lower left - Exiting the trigger mode.

[0167] Entering the channel selection mode - Moving a single hand in a clockwise rotation (or any other suitable gesture)

[0168] Channel selection mode activated:

[0169] Moving a single hand vertically downward - Moving to the next channel in the instrument;

[0170] Moving a single hand vertically upward - Moving to the previous channel in the instrument;

[0171] Moving a single hand horizontally from left to right - Connecting the selected channel;

[0172] Moving a single hand horizontally from right to left - Disconnecting the selected channel;

[0173] Moving a single hand in a counterclockwise rotation - Exiting the channel selection mode.

[0174] Entering the cursor control mode - Moving a single hand diagonally from upper left to lower right (or any other suitable gesture)

[0175] Cursor control mode activated:

[0176] Moving a single hand in a clockwise rotation - Entering the channel selection mode. Operate in this mode using the following rules to select the channel to which the cursor is to be applied;

[0177] Moving a single hand diagonally from upper left to lower right - Switching the cursor between vertical and horizontal modes;

[0178] Moving a single hand diagonally from lower left to upper right - Switching the active cursor;

[0179] Moving a single hand horizontally from left to right - Moving the active cursor backward on the waveform;

[0180] Moving a single hand horizontally from right to left - Moving the active cursor forward on the waveform;

[0181] Moving a single hand vertically upward - Moving the active cursor in the positive direction in the waveform;

[0182] One - hand downward vertical movement - Move the active cursor in the negative direction along the waveform;

[0183] One - hand diagonal movement from bottom - right to top - left - Exit the cursor control mode.

[0184] Enter the general menu mode - One - hand counter - clockwise rotation movement

[0185] Menu activation:

[0186] One - hand upward / downward vertical movement - Scroll the menu up / down;

[0187] One - hand horizontal movement from left to right - Select the current menu option to activate the option or open a sub - menu;

[0188] One - hand horizontal movement from right to left - Exit the menu or activate a sub - menu.

[0189] In an integrated solution, the gesture capture mechanism, the gesture interpretation module, and the test and measurement instrument and all associated therewith can be a single unit. The remote unit can be implemented using something as simple as a smart phone, which uses a camera in conjunction with an application that interprets gestures and issues commands to the instrument. The application can also provide feedback to the user on the screen (e.g., command identifiers, command sequences, etc.).

[0190] Examples

[0191] Illustrative examples of the technology disclosed herein are provided below. Embodiments of the technology can include any one or more of the examples described below and any combination thereof.

[0192] Example 1 relates to a test and measurement system, which includes: a data store configured to store data for enhancement settings for dynamically enhancing a physical test environment; and a computing device coupled to the data store, the computing device being configured to receive an input feed from the physical test environment, create an enhanced image based on the enhancement settings and the input feed, and output the enhanced image to be overlaid on the physical test environment to enhance the user's view of the physical test environment.

[0193] Example 2 includes the subject matter of Example 1, further including a display device configured to visually present the enhanced image overlaid on the physical test environment to the user.

[0194] Example 3 includes the subject matter of Example 2, wherein the display device is a head - mounted display, a tablet device, an image display screen, or a display of a mobile phone.

[0195] Example 4 includes the subject matter of any one of Examples 1 to 3, and further includes a camera configured to provide an input feed.

[0196] Example 5 includes the subject matter described in Example 4, wherein the camera is integrated in a display device.

[0197] Example 6 includes the subject matter of Example 2, wherein the display device is at least partially transparent.

[0198] Example 7 includes the subject matter of any one of Examples 1 to 6, wherein the enhanced image includes content captured by a camera coupled to a measurement probe.

[0199] Example 8 includes the subject matter of any one of Examples 1 to 7, wherein the enhanced image includes generated waveforms related to a device under test in a physical test environment.

[0200] Example 9 includes the subject matter of any one of Examples 1 to 8, wherein the enhanced image includes a virtual user interface.

[0201] Example 10 includes the subject matter of Example 9, wherein the virtual user interface includes instructions for the user.

[0202] Example 11 includes the subject matter of Example 10, wherein the instructions include an indication of an area to be tested or an area to be avoided.

[0203] Example 12 includes the subject matter of any one of Examples 10 to 11, and further includes an input device, wherein the computing device is further configured to change the instructions in response to input received from a user via the input device.

[0204] Example 13 includes the subject matter of any one of Examples 1 to 12, and further includes an input device, wherein the computing device is further configured to change the enhanced image in response to input received from a user via the input device.

[0205] Example 14 includes the subject matter of Example 7, wherein the computing device is further configured to place a virtual marker within the enhanced image in response to the user causing the measurement probe to physically contact a specific part in the physical test environment, the virtual marker corresponding to the identified location of the specific part.

[0206] Example 15 includes the subject matter of any one of Examples 1 to 14, wherein the enhancement settings are configurable to allow a user to select a location within the display device for positioning the enhanced image.

[0207] Example 16 relates to a method of machine control, which includes: using a data store to store enhancement settings for dynamically enhancing a physical test environment; receiving an input feed from the physical test environment; using a computing device to create an enhanced image based on the enhancement settings and the input feed; and outputting the enhanced image to be superimposed on the physical test environment to enhance the user's view of the physical test environment.

[0208] Example 17 relates to one or more computer-readable storage media, which include instructions that, when executed by a processor, cause the processor to: receive enhancement settings for dynamically enhancing a physical test environment from a data store; receive an input feed from the physical test environment; generate an enhanced image based on the enhancement settings and the input feed; and output the enhanced image to be superimposed on the physical test environment to enhance the user's view of the physical test environment.

[0209] Example 18 includes the subject matter of Example 17, wherein the enhanced image includes: content captured by a camera coupled to a measurement probe, and generated waveforms related to a device under test in the physical test environment.

[0210] Example 19 includes the subject matter of any one of Examples 17 to 18, wherein the enhanced image includes a virtual user interface, and the virtual user interface includes instructions for the user, and the instructions include indications of areas to be tested or areas to be avoided.

[0211] Example 20 relates to a test and measurement system, which includes: a device under test (DUT); a data store configured to store enhancement settings for dynamically enhancing a physical test environment; a head-mounted display device configured to be worn by a user; a measurement probe configured to interact with the DUT; a probe camera coupled to the measurement probe; and a computing device coupled to the data store, the computing device being configured to receive an input feed from the physical test environment, receive an image captured by the camera coupled to the measurement probe, create an enhanced image based on the enhancement settings, the input feed, and the image captured by the probe camera, and cause the head-mounted display device to visually present the enhanced image superimposed on the physical test environment to the user.

[0212] Aspects of the present invention may operate on specially created hardware, firmware, digital signal processors, or specially programmed computers, the specially programmed computers including processors operating according to programming instructions that may be stored in one or more memory instances or on one or more computer-readable media. As used herein, the term "controller" or "processor" is intended to include microprocessors, microcomputers, application specific integrated circuits (ASICs), and dedicated hardware controllers. One or more aspects of the present invention may be embodied in computer-usable data and computer-usable instructions executed by one or more computers or other devices, such as in one or more program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., which perform particular tasks or implement particular abstract data types when executed by a processor in a computer or other device. Computer-executable instructions may be stored on computer-readable media such as hard disks, optical disks, removable storage media, solid state memories, random access memories (RAM), etc. As will be appreciated by those skilled in the art, the functions of program modules may be combined or distributed as desired in various aspects. Additionally, the functions may be embodied in whole or in part in firmware or hardware equivalents, such as integrated circuits and FPGAs, etc. Certain data structures may be used to more effectively implement one or more aspects of the present invention, and such data structures are contemplated within the scope of the computer-executable instructions and computer-usable data described herein.

[0213] Aspects of the present disclosure admit of various modifications and alternative forms. Specific aspects have been shown by way of example in the drawings and described in detail above. However, it should be noted that the examples disclosed herein are presented for purposes of discussion clarity and are not intended to limit the scope of the general concepts disclosed herein to the specific aspects described herein unless explicitly restricted. Accordingly, the present disclosure is intended to cover all modifications, equivalents, and alternatives of the aspects described in accordance with the accompanying drawings.

[0214] References in the specification to aspects, examples, etc. indicate that the described item may include a particular feature, structure, or characteristic. However, each disclosed aspect may or may not include the particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same aspect unless specifically indicated. Further, when a particular feature, structure, or characteristic is described in connection with a particular aspect, such feature, structure, or characteristic may be employed in connection with other disclosed aspects whether or not such feature is explicitly described in connection with such other disclosed aspects.

[0215] In some cases, the disclosed aspects may be implemented in hardware, firmware, software, or any combination thereof. The disclosed aspects may also be implemented as instructions carried or stored on one or more computer-readable media, which may be read and executed by one or more processors. Such instructions may be referred to as a computer program product. As discussed herein, a computer-readable medium means any medium that can be accessed by a computing device. By way of example and not limitation, a computer-readable medium may include computer storage media and communication media. Computer storage media means any medium that can be used to store computer-readable information. By way of example and not limitation, computer storage media may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, and any volatile or nonvolatile, removable or non-removable medium implemented in any technology. Computer storage media does not include signals per se and transient forms of signal transmission.

[0216] Communication media means any medium that can be used for communication of computer-readable information. By way of example and not limitation, communication media may include coaxial cable, fiber optic cable, air, or any other medium suitable for communication of electrical, optical, radio frequency (RF), infrared, acoustic, or other types of signals.

[0217] The principles of the present invention have been described and illustrated with reference to the illustrated embodiments. It will be appreciated that the illustrated embodiments may be modified in arrangement and detail and combined in any desired manner without departing from such principles. Also, while the foregoing discussion has focused on specific embodiments, other configurations are also contemplated.

[0218] In particular, even though expressions such as "embodiments according to the present invention" are used herein, these expressions are meant to generally refer to the possibility of embodiments and are not intended to limit the present invention to a specific embodiment configuration. As used herein, these terms may refer to the same or different embodiments that may be combined into other embodiments.

[0219] Accordingly, in view of the various permutations of the embodiments described herein, this detailed description and the accompanying materials are only intended to be illustrative and should not be regarded as limiting the scope of the present invention. Accordingly, the present invention is claimed to cover all such modifications that may fall within the scope and spirit of the following claims and their equivalents.

Claims

1. A test and measurement system, comprising: a data store configured to store enhancement settings for dynamically enhancing a physical test environment; and a computing device coupled to the data store, the computing device being configured to: receive an input feed from the physical test environment; create an enhanced image based on the enhancement settings and the input feed; and output the enhanced image to be overlaid on the physical test environment to enhance a user's view of the physical test environment, wherein a user is enabled to view a waveform or its measurement values in three-dimensional space and move in three-dimensional space to view the waveform or its measurement values from different perspectives, and wherein a user is enabled to manipulate the perspective of the enhanced image as if the user were holding a three-dimensional measurement.

2. The test and measurement system according to claim 1, further comprising a display device configured to visually present the enhanced image overlaid on the physical test environment to a user.

3. The test and measurement system according to claim 2, wherein the display device is a head-mounted display, a tablet device, or a display of a mobile phone.

4. The test and measurement system according to claim 2, wherein the display device is an image display screen.

5. The test and measurement system according to claim 2, further comprising a camera configured to provide the input feed.

6. The test and measurement system according to claim 5, wherein the camera is integrated in the display device.

7. The test and measurement system according to claim 2, wherein the display device is at least partially transparent.

8. The test and measurement system according to claim 5, wherein the enhanced image includes content captured by a camera coupled to a measurement probe.

9. The test and measurement system according to claim 1, wherein the enhanced image includes a generated waveform related to a device under test in the physical test environment.

10. The test and measurement system according to claim 1, wherein the enhanced image includes a virtual user interface.

11. The test and measurement system according to claim 10, wherein the virtual user interface includes instructions for the user.

12. The test and measurement system according to claim 11, wherein the instructions include an indication of an area to be tested or an area to be avoided.

13. The test and measurement system according to claim 11, further comprising an input device, wherein the computing device is further configured to change the instructions in response to an input received from the user via the input device.

14. The test and measurement system according to claim 1, further comprising an input device, wherein the computing device is further configured to change the enhanced image in response to an input received from the user via the input device.

15. The test and measurement system according to claim 8, wherein the computing device is further configured to place a virtual marker within the enhanced image corresponding to an identified location of the specific part in response to the user causing the measurement probe to physically contact a specific part in the physical test environment.

16. The test and measurement system according to claim 2, wherein the enhancement settings are configurable to allow a user to position a location for an enhanced image within a display device.

17. A method of machine control, comprising: using a data store to store enhancement settings for dynamically enhancing a physical test environment; receiving an input feed from the physical test environment; using a computing device to create an enhanced image based on the enhancement settings and the input feed; and outputting the enhanced image to be overlaid on the physical test environment to enhance a user's view of the physical test environment, wherein the user is enabled to view a waveform or its measurement values in three-dimensional space and move in three-dimensional space to view the waveform or its measurement values from different perspectives, and wherein the user is enabled to manipulate the perspective of the enhanced image as if the user were holding a three-dimensional measurement.

18. One or more computer-readable storage media, comprising instructions that, when executed by a processor, cause the processor to: receive enhancement settings for dynamically enhancing a physical test environment from a data store; receive an input feed from the physical test environment; generate an enhanced image based on the enhancement settings and the input feed; and output the enhanced image to be overlaid on the physical test environment to enhance a user's view of the physical test environment, wherein the user is enabled to view a waveform or its measurement values in three-dimensional space and move in three-dimensional space to view the waveform or its measurement values from different perspectives, and wherein the user is enabled to manipulate the perspective of the enhanced image as if the user were holding a three-dimensional measurement.

19. The one or more computer-readable storage media according to claim 18, wherein the enhanced image comprises: content captured by a camera coupled to a measurement probe; or a generated waveform associated with a device under test in a physical test environment.

20. The one or more computer-readable storage media according to claim 18, wherein the enhanced image comprises a virtual user interface, the virtual user interface comprising guidance to the user, the guidance comprising an indication of an area to be tested or an area to be avoided.

21. A test and measurement system, comprising: a device under test; a data store configured to store enhancement settings for dynamically enhancing a physical test environment; a head-mounted display device configured to be worn by a user; a measurement probe configured to interact with the device under test; and a probe camera coupled to the measurement probe; and a computing device coupled to the data store, the computing device configured to: receive an input feed from the physical test environment; receive an image captured by a camera coupled to the measurement probe; create an enhanced image based on the enhancement settings, the input feed, and the image captured by the probe camera; and cause the head-mounted display device to visually present the enhanced image overlaid on the physical test environment to the user. whereby a user is enabled to view a waveform or its measured values in a three-dimensional space and move in the three-dimensional space to view the waveform or its measured values from different perspectives, and whereby a user is enabled to manipulate the perspective of the enhanced image as if the user were holding the three-dimensional measured values.

Images