Virtual reality training simulator

Abstract

A virtual reality (VR) system is provided for training a user to assemble a device in a virtual work area. The VR system sequentially displays a list of ordered steps by highlighting tools and locations in the virtual work area. The highlighted tool and position indicates movement or processing of the component to be performed in the current step. In one embodiment, a VR system includes a haptic glove for providing haptics to a user. A haptic glove is used to train a user to detect problems (e.g., burrs) of surface finish of components during assembly.

Description

Technical Field

[0001] The present disclosure relates generally to virtual reality, and more particularly to virtual reality based training of hydraulic pump assemblies. Background Art

[0002] Learning how to perform the steps required to assemble a hydraulic pump requires many hours of instruction, training, and practice. For example, the trainee needs to learn the order in which the components of the hydraulic pump are assembled, the surface treatment of the components (e.g., oiling), the machining of the components (e.g., filing), and how to operate the equipment required to assemble the hydraulic pump.

[0003] Typically, learning to assemble a hydraulic pump requires a trainee to operate a real workstation by performing the steps of assembling the hydraulic pump using real parts. Such real-world training may take up scarce resources including workspace, tools, and parts. In addition, training typically requires an instructor to teach the trainee, which takes up the instructor's time that might otherwise be assembling the hydraulic pump. Summary of the invention

[0004] The present disclosure provides a virtual reality (VR) system for training a user to assemble equipment in a virtual work area by sequentially displaying a list of ordered steps that visually indicates the next step by highlighting tools and locations in the virtual work area. In one embodiment, the VR system includes a haptic glove for providing a sense of touch to the user to indicate the surface finish of a component of an assembly.

[0005] Although many features are described herein with respect to embodiments of the invention; however, features described with respect to a given embodiment may also be used in conjunction with other embodiments. The following description and accompanying drawings set forth certain illustrative embodiments of the invention. However, these embodiments illustrate only a few of the various ways in which the principles of the invention may be employed. Other objects, advantages, and novel features according to various aspects of the invention will become apparent from the detailed description discussed below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The drawings, which are not necessarily to scale, illustrate various aspects of the present invention, wherein like reference numbers are used to indicate the same or similar parts in different views.

[0007] Figure 1 An exemplary embodiment of a virtual reality system is shown.

[0008] Figure 2 An exemplary training space displayed via a virtual reality system is shown.

[0009] Figure 3 yes Figure 2An enlarged view of a workbench in a training space in FIG. 1 showing components of an assembly.

[0010] Figure 4 yes Figure 2 A magnified view of the workbench in FIG. 1 , showing a sub-assembly of tools and components.

[0011] Figure 5 yes Figure 2 A magnified view of the workbench in Figure 1, showing two highlighted components.

[0012] Figure 6 yes Figure 2 A magnified view of the workbench in FIG. 1 showing a highlighted tool and a part to be processed using the highlighted tool.

[0013] Figure 7 is an exemplary view of an augmented reality system showing a graphical overlay on a user's environment.

[0014] Figure 8 Only shows Figure 7 A view of a graphic overlay in .

[0015] Fig. 9 An exemplary method for training a user to assemble a device using a virtual reality system is provided.

[0016] The present invention is described in detail below with reference to the accompanying drawings. In the accompanying drawings, each element with a reference number is similar to other elements with the same reference number, regardless of the letter label after the reference number. In this article, a reference number with a specific letter label after the reference number refers to the specific element with the number and letter label, while a reference number without a specific letter label refers to all elements with the same reference number, regardless of the letter label after the reference number. DETAILED DESCRIPTION

[0017] According to a general embodiment, a virtual reality (VR) system is provided for training a user to assemble a device in a virtual work area. The VR system sequentially displays an ordered list of steps by highlighting virtual tools and locations in the work area. The highlighted tools and locations indicate the movement or processing of the component to be performed in the current step. In one embodiment, the VR system includes a haptic glove for providing a sense of touch to the user. The haptic glove is used to train the user to detect problems (e.g., burrs) in the surface finish of the component during the assembly process.

[0018] Go to Figure 1, a virtual reality system 10 for training a user to assemble a device 12 is shown. The virtual reality system 10 includes a display 14, a hand sensor 16, a memory 18, and a processor circuit 20. The display 14 displays a three-dimensional (3D) training space 22. The training space 22 includes a work area 24, the work area 24 having a workbench 26, a set of tools 28 arranged in the work area 24, and a part 30 arranged in the work area 24. The hand sensor 16 detects the position of the user's hand 12. The memory 18 stores an assembly set 32 ​​and an instruction set 34. The processor circuit 20 generates the training space 22 and causes the display 14 to display the training space 22. Figure 2 An exemplary training space 22 is shown.

[0019] Those skilled in the art will appreciate that the memory 18 (also referred to as a computer-readable medium) may be, for example, a cache, a flash memory, a hard disk, a removable storage medium, a volatile memory, a non-volatile memory, a random access memory (RAM), or one or more of other suitable devices. In a typical configuration, the memory 18 may include a non-volatile memory for long-term data storage and a volatile memory used as a system memory for the processor 18. The memory 18 may exchange data with the circuit via a data bus. There may also be accompanying control lines and an address bus between the memory 18 and the circuit. The memory 18 is considered to be a non-transitory computer-readable medium.

[0020] The training space 22 may be any virtual space suitable for training a user to assemble an assembly set 32. For example, the training space 22 may be a three-dimensional (3D) virtual room large enough to include a workbench 26, tools 28, and parts 30. The work area 24 of the training space 22 may be an area of ​​the training space that includes the workbench 26, tools 28, and parts 30. The training space 22 may include one or more work areas 24 (e.g., multiple copies of the same work area).

[0021] The processor circuit 20 receives a selection of an assembly 36 from the set of assemblies 32. Each assembly 36 in the set of assemblies 32 is formed by combining assembly parts 38. The assembly parts 38 are subsets of the components 30 arranged in the work area 24.

[0022] Parts 38 may be a subset of components 30 arranged in work area 24. That is, work area 24 may not include any components that are not used to assemble the selected assembly. Alternatively, parts 38 may be a subset of components 30 such that there are additional components that are not included in selected assembly 36. Components 30 may be virtual (also referred to as digital) representations of parts used to assemble an assembly in the real world.

[0023] Assembly 32 may be any suitable device that is at least partially manually assembled. For example, assembly 36 may include a hydraulic pump and / or a hydraulic motor.

[0024] The processor circuit 20 receives an instruction set 40 for the selected assembly 36. Each assembly 36 is associated with an instruction set 40 from the instruction set 34. Each instruction set 40 in the instruction set 34 includes an ordered list of steps 42 that are executed to assemble the associated assembly 36 by combining the parts 38 of the associated assembly. Each step 44 in the ordered list of steps 42 produces a sub-assembly 46 of the assembly 36 by (1) adding at least one of the parts 38 to the sub-assembly 46 or (2) processing at least one of the parts 38 of the assembly using at least one tool in the set of tools 28. Each step 44 in the ordered list of steps 42 also identifies at least one of a next part 38 to be added to the sub-assembly 46, a placement location 48 for the next part 38, or a next tool 28 for processing at least one of the parts 38 of the assembly.

[0025] Tools 28 may be virtual representations of real-world tools used to assemble assemblies. For example, tools may include vises, clamps, files, rasps, presses, drills, lubricant dispensers (e.g., oil), etc. The tools 28 in the work area 24 may be only those required to assemble the selected assembly. Alternatively, the work area 24 may include additional tools 28 that are not required to prepare the selected assembly.

[0026] The processor circuit 20 causes the display 14 to sequentially display the ordered step list 42 of the received instruction set 40. The display of the steps includes displaying the current step 40 in the ordered step list 42 by visually highlighting the next object in the training space. The next object 50 is the next part 38 or the next tool 28. For example, in Figure 3 , three parts 38a, 38b, 38c are shown on the workbench 26. Part 38c is highlighted (e.g., using an outline) to indicate that part 38c is the next object.

[0027] The next object 50 in the training space 22 may be highlighted using any suitable visual effect for distinguishing the next object 50 from other objects 50 in the training space 22. For example, the next object 50 may be visually highlighted by changing the color or brightness of the next object 50 so that the next object 50 is visually distinguished from both other tools in the set of tools 28 and other components 30 arranged in the work area 24.

[0028] The display of the step also includes detecting the user's interaction with the next highlighted object 50 using the hand sensor 16 based on the mapping of the measured position 52 of the user's hand 12 to the training space 22. For example, in Figure 3 , a hand icon 54 is shown to indicate that the user's hand 12 is mapped to the training space 22. The hand icon 54 may serve as visual feedback to the user, allowing the user to identify the position of his hand 12 in the virtual training space 22.

[0029] The hand sensor 16 may be any suitable device for sensing the position of the user's hand 12. For example, the hand sensor 16 may be a device (e.g., a glove, a controller, etc.) held or worn by the user's hand 12 for detecting the position of the user's hand 12. Alternatively or additionally, the hand sensor 16 may include a camera (e.g., attached to a headset worn by the user) for visually detecting the user's hand 12.

[0030] After detecting that the user interacts with the highlighted next object 50, the processor circuit 20 highlights a target location 56 for placing or interacting with the highlighted next object 50. The target location includes a location in a work area, a tool in a set of tools, or a part of an assembly. For example, go to Figure 4 , the user picked up Figure 3 After picking up the part 38c, a target location 56 for placing the part 38c is shown. The target location 56 can be highlighted in any suitable distinguishing manner. For example, in Figure 4 , a partially transparent version of part 38c is shown where part 38c will be placed.

[0031] When the user completes the current step, the processor circuit 20 repeatedly selects the next step 44 as the current step 44 in the ordered step list 42 and displays the current step 44 in the ordered step list 42 until all steps 44 in the instruction set 40 are completed, thereby assembling the associated assembly 36.

[0032] For example, in Figure 4 After the steps shown, the next step can be as follows Figure 5 , wherein the next two parts 38 are highlighted. Once the user has placed the two parts 38 into the subassembly 46 (not shown), the next step may be shown. Figure 6 In the next step shown, the next object 50 is a tool 28. Figure 6 In the example, the target location 56 is a part of the subassembly 46. That is, the target location 56 is a part of the subassembly 46 that needs to be processed. In this example, the file is applied to the surfaces of the two parts 38.

[0033] The processor circuit 20 may have a variety of implementations. For example, the processor circuit 20 may include any suitable device, such as a processor (e.g., a CPU), a programmable circuit, an integrated circuit, a memory and I / O circuit, an application-specific integrated circuit, a microcontroller, a complex programmable logic device, other programmable circuits, etc. The processor circuit 20 may also include a non-transitory computer-readable medium (e.g., a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory)), or any other suitable medium. Instructions for performing the methods described below may be stored in a non-transitory computer-readable medium and executed by the processor circuit 20. The processor circuit 20 may be communicatively coupled to the computer-readable medium and the network interface via a system bus, a motherboard, or using any other suitable structure known in the art.

[0034] The display 14 may be any suitable device for displaying the training space 22 to the user. For example, the display 14 may be a head mounted device, such as a standard virtual reality device (e.g., Meta Quest, PlayStation VR, Oculus Rift, HTC VIVE, etc.). The processor circuit 20 may be included in the same headset as the display 14. Alternatively or additionally, the processor circuit 20 may be located separately from the display 14 and communicate with the display 14 by wire or wireless means.

[0035] The virtual reality system 10 may include a haptic glove 58. The haptic glove 58 provides tactile feedback to the hand 12 of the user wearing the haptic glove. The tactile feedback is based on the characteristics of the object with which the user interacts in the training space 22. For example, the next object 50 that is highlighted may be a part 38 of an assembly, and the highlighted target location 56 may be a highlighted portion of the next object 50 that is highlighted. That is, in step 40 of the instruction set, a portion of the object 50 may be highlighted without highlighting the location where the object 50 is to be placed or the tool 28 for handling the object 50. Instead, the user may interact with the highlighted portion by moving the user's hand 12 to the highlighted portion. When the user interacts with the highlighted portion, the processor circuit 20 may cause the haptic glove 58 to provide tactile feedback to the user indicating the surface characteristics of the highlighted portion. The surface characteristics may be any suitable characteristics of the object. For example, the surface characteristics may be roughness, texture, hardness, temperature, etc.

[0036] For example, the surface characteristic may include the presence of burrs on the highlighted display portion. That is, when the user interacts with the highlighted portion and the highlighted portion has burrs, the processor circuit 20 may cause the haptic glove 58 to provide tactile feedback indicating the presence of burrs. For example, the tactile feedback may cause the user's hand 12 to feel as if the user is running their hand over a surface with bumps (i.e., as if the user is running her hand over burrs on the surface).

[0037] When the user feels the burr on the highlighted portion, the processor circuit 20 may highlight the surface treatment tool for removing the burr among the tools 28 on the display as the next object 50. The processor circuit 20 may also highlight the highlighted portion including the burr as the target position 56 on the display.

[0038] The haptic glove 58 may be any suitable device for providing haptic (also referred to as touch) feedback to the user. For example, the haptic glove 58 may simulate the characteristics of a virtual object (e.g., texture, weight, hardness, etc.).

[0039] Go to again Figure 6 , the highlighted next object may be a tool 28, and the target location 56 may include the highlighted part 38 of the sub-assembly. The user may interact with the highlighted part 38 of the sub-assembly using the highlighted next object 50 by moving the highlighted next object 50 near the target location 56. The user may interact with the highlighted part 38 of the sub-assembly, resulting in processing of the highlighted part based on the highlighted next object.

[0040] like Figure 5 and Figure 6 As shown in the transition between, the next highlighted object 50 may be the next part 38, and the target location 56 may include the highlighted part 38 of the sub-assembly. The user may interact with the highlighted part 38 of the sub-assembly using the next highlighted object 50 by moving the next highlighted object 50 near the target location 56. The user's interaction with the highlighted part 38 of the sub-assembly may cause the next highlighted object 50 to be added to the sub-assembly 46.

[0041] The training space 22 may include a manual 60 for providing the user with access to an ordered list of steps 42 of an instruction set for a selected assembly 36. For example, the manual 60 may be in the form of a tablet computer (e.g., an iPad), and the steps 44 of the instruction set 40 may be displayed in text form on the manual 60, while the steps 44 are also illustrated by highlighting parts and locations in the training space 22 as described above.

[0042] In one embodiment, as the user's assembly proficiency of a particular assembly 36 increases, the difficulty of the training can be increased. At a simple difficulty level, the steps can be displayed visually as described above. At a higher difficulty level, the steps can be displayed only by using the manual 60.

[0043] In one embodiment, processor circuit 20 also receives instructions from the user to move in training space 22 so that the user's viewpoint in training space is changed from a previous viewpoint to a new viewpoint. Processor circuit 20 displays training space 22 including displaying training space 22 from a new viewpoint in training space. For example, the user may use gestures to move from one area of ​​training space 22 to another area of ​​training space 22. As the user moves in training space 22, the viewpoint shown in the display may change, just as if the user were actually moving in training space 22.

[0044] In one embodiment, the virtual reality system 10 is an augmented reality system. That is, the display 14 shows a view of the user's environment 62 including a graphic overlay 64. That is, the display 14 shows the user's real environment 62 with the graphic overlay 64, rather than showing a completely virtual space. In this embodiment, the processor circuit 20 can identify (also referred to as mapping) the position and outline of a workbench in a work area of ​​the environment, a set of tools arranged in the work area, and components arranged in the work area. That is, the processor circuit 20 identifies physical objects in the user's environment 62, rather than identifying visually generated objects. In particular, the processor circuit 20 identifies the position and outline (e.g., shape) of the object. The processor circuit 20 can use any suitable method to identify the object. For example, the processor circuit 20 can use visual object recognition from a camera that is positioned to image the work area. Alternatively or additionally, the object can include an identification mark (e.g., a barcode, an infrared dot, a serial number, etc.) to help identify the object.

[0045] Go to Fig. 9, shows an exemplary method 100 for training a user to assemble a device using a virtual reality system 10 having a display 14, a processor circuit 20, and a hand sensor 16. In step 102, the processor circuit 20 generates a three-dimensional (3D) training space 22. In step 104, the training space 22 is displayed to the user using the display 14. In step 106, a selection of an assembly 36 is received. For example, the processor circuit 20 may receive the selected assembly 36 from the user via user input, or the processor circuit 20 may select the assembly 36 based on a training program selected for the user. In step 108, the processor circuit 20 receives an instruction set 40 for the selected assembly 36 from the instruction set 34. For example, the processor circuit 20 may retrieve the instruction set 40 from the memory 18. In step 110, the processor circuit sequentially causes the ordered list of steps of the received instruction set to be displayed on the display.

[0046] As described above, the augmented reality system may display a sequential list of steps for assembling an assembly. In this embodiment, the processor circuit 20 visually highlights the next object 50 in the graphic overlay 64. For example, the processor circuit 20 may generate an indicator 66 outlining the outline of the next object 50 in the graphic overlay 64. Similarly, the processor circuit 20 may generate an indicator 66 outlining a target location in the graphic overlay 64 for placing the highlighted next object or for interacting with the highlighted next object. Figure 7 An exemplary display showing a user's environment 62 and a graphical overlay 64 is depicted. Figure 8 Only shown Figure 8 Graphic overlay in .

[0047] All range and ratio limits disclosed in the specification and claims may be combined in any manner. Unless expressly stated otherwise, references to "a", "an" and / or "the" may include one or more than one, and references to items in the singular may also include the items in the plural.

[0048] Although the present invention has been shown and described with respect to a certain embodiment or certain embodiments, those skilled in the art will appreciate equivalent modifications and changes after reading and understanding this specification and the accompanying drawings. In particular, with respect to the various functions performed by the above-mentioned elements (components, assemblies, devices, compositions, etc.), unless otherwise indicated, the terms used to describe these elements (including references to "devices") are intended to correspond to any element that performs the specified function of the element (i.e., functionally equivalent), even if the structure is not equivalent to the disclosed structure that performs the function in the exemplary embodiment or multiple exemplary embodiments of the present invention shown herein. In addition, although a feature of the present invention may be described only for one or more of the multiple illustrated embodiments, the feature may be combined with one or more other features of other embodiments to meet the needs of any given or specific application and to facilitate any given or specific application.

Claims

1. A method for training a user to assemble a device using a virtual reality system, the virtual reality system having a display, a processor circuit, and a hand sensor, the method comprising: generating a three-dimensional (3D) training space using the processor circuit of the virtual reality system, wherein the training space includes a work area having a workbench, a set of tools disposed in the work area, and components disposed in the work area; displaying the training space to a user using the display; Receives a selection of an assembly from a collection of assemblies, where: Each assembly in the assembly set is formed by combining parts of the assembly; and and The parts of the assembly are a subset of the components arranged in the work area; The processor circuit receives an instruction set for a selected assembly from a set of instructions, wherein: Each assembly in the set of assemblies is associated with a set of instructions from the set of instructions; Each of the sets of instructions includes an ordered list of steps that are executed to assemble an associated assembly by combining parts of the assembly; Each step in the ordered list of steps: The subassembly of the assembly is produced by at least one of the following: adding at least one of said parts to said subassembly; or processing at least one of the parts of the assembly using at least one tool from the set of tools; Identify at least one of the following: a next part of said parts for adding to said assembly of said subassembly; the placement location of the next part; or a next tool in the set of tools for processing at least one of the parts of the assembly; The processor circuit sequentially causes a list of ordered steps of the received instruction set to be displayed on a display, comprising: Display the current step in the ordered list of steps by: visually highlighting a next object in the training space, wherein the next object is a next part or a next tool; detecting user interaction with the next highlighted object using the hand sensor; and After detecting a user interaction with the highlighted next object, highlighting a target location for placing the highlighted next object or for interacting with the highlighted next object, wherein the target location comprises a location in the work area, a tool in the set of tools, or a part of the assembly; and When the user completes the current step, the next step is repeatedly selected as the current step in the ordered step list, and the current step in the ordered step list is displayed until all steps in the instruction set are completed, thereby assembling the associated assembly parts.

2. The method according to claim 1, wherein: The virtual reality system includes a haptic glove configured to provide tactile feedback to a hand of a user wearing the haptic glove based on characteristics of an object with which the user interacts in the training space; The next object highlighted is a part of the assembly; The target position of the highlight is a highlighted portion of the next object of the highlight; a user interacting with the highlighted portion by moving a user's hand to the highlighted portion; as well as When the user interacts with the highlighted portion, the haptic glove provides tactile feedback to the user indicative of surface characteristics of the highlighted portion.

3. The method according to claim 2, wherein: The surface characteristics include the presence of burrs on the highlighted portion; as well as When a user interacts with the highlighted portion and the highlighted portion includes a glitch: The haptic glove provides tactile feedback indicating the presence of a burr; highlighting a surface treatment tool for removing burrs among the tools as the next object; and The highlighted portion including the burr is highlighted as the target position.

4. A method according to claim 1 or any one of the preceding claims, wherein: The next object highlighted is the tool; The target location includes a highlighted part of the subassembly; and a user interacting with a highlighted part of the subassembly using the highlighted next object by moving the highlighted next object proximate to the target location; and User interaction with a highlighted part of the subassembly causes processing of the highlighted part based on the highlighted next object.

5. The method of claim 1 or any one of the preceding claims, wherein: The next object highlighted is the next part; The target location includes a highlighted part of the subassembly; and the user interacts with the highlighted part of the sub-assembly using the highlighted next object by moving the highlighted next object proximate to the target location; and User interaction with a highlighted part of the sub-assembly causes the highlighted next object to be added to the sub-assembly.

6. A method according to claim 1 or any one of the preceding claims, wherein: The training space includes a manual for providing access to an ordered list of steps for a set of instructions for a selected assembly.

7. A method according to claim 1 or any one of the preceding claims, wherein: Visually highlighting the next object in the training space includes changing a color or brightness of the next object so that the next object is visually distinguished from other tools in the set of tools and from other components arranged in the working area.

8. The method of claim 1 or any one of the preceding claims, further comprising: A user is provided with controls to move in the training space such that a user's viewpoint in the training space changes from a previous viewpoint to a new viewpoint, wherein displaying the training space by the processor circuit includes displaying the training space from the new viewpoint of the training space.

9. A virtual reality system for training a user to assemble a device, the virtual reality system comprising: a display configured to display a three-dimensional (3D) training space, wherein the training space includes a work area having a workbench, a set of tools arranged in the work area, and components arranged in the work area; a hand sensor configured to detect a position of a user's hand; A memory configured to store: A collection of assemblies, where: Each assembly in the set of assemblies is formed by combining parts of the assembly; and The parts of the assembly are a subset of the components arranged in the work area; and A set of instructions, where: Each assembly in the set of assemblies is associated with a set of instructions from the set of instructions; Each of the sets of instructions includes an ordered list of steps that are executed to assemble an associated assembly by combining parts of the assembly; and Each step in the ordered list of steps: creating a sub-assembly of the assembly by at least one of: adding at least one of the parts to the sub-assembly, or processing at least one of the parts of the assembly using at least one tool from the set of tools; and identifying at least one of: a next part in the parts of the assembly of the subassembly, a location for placement of the next part, or a next tool in the set of tools for processing at least one of the parts of the assembly; and A processor circuit, the processor circuit being configured to: generating the training space and causing the display to display the training space; receiving a selection of an assembly from the set of assemblies; receiving a set of instructions for a selected assembly from the set of instructions; and The sequential step list of the received instruction set is displayed on the display in sequence, including: Display the current step in the ordered list of steps by: visually highlighting a next object in the training space, wherein the next object is a next part or a next tool; detecting, using the hand sensor, a user interaction with the highlighted next object based on mapping the measured position of the user's hand into the training space; and After detecting a user interaction with the highlighted next object, highlighting a target location for placing the highlighted next object or for interacting with the highlighted next object, wherein the target location comprises a location in the work area, a tool in the set of tools, or a part of the assembly; and When the user completes the current step, the next step is repeatedly selected as the current step in the ordered step list, and the current step in the ordered step list is displayed until all steps in the instruction set are completed, thereby assembling the associated assembly parts.

10. The virtual reality system of claim 9, further comprising a haptic glove configured to provide tactile feedback to a hand of a user wearing the haptic glove based on characteristics of an object with which the user interacts in the training space, wherein: The next object highlighted is a part of the assembly; The target position of the highlight is a highlighted portion of the next object of the highlight; a user interacting with the highlighted portion by moving a user's hand to the highlighted portion; as well as When the user interacts with the highlighted portion, the processor circuit causes the haptic glove to provide tactile feedback to the user indicative of surface characteristics of the highlighted portion.

11. The virtual reality system according to claim 10, wherein: The surface characteristics include the presence of burrs on the highlighted portion; as well as When a user interacts with the highlighted portion and the highlighted portion includes a glitch, the processor circuit: causing the haptic glove to provide tactile feedback indicating the presence of a burr; highlighting a surface treatment tool for removing burrs among the tools as the next object on the display; as well as The highlighted portion including the glitch is highlighted as the target position on the display.

12. A virtual reality system according to claim 9 or any one of claims 9 to 11, wherein: The next object highlighted is the tool; The target location includes a highlighted part of the subassembly; and a user interacting with a highlighted part of the subassembly using the highlighted next object by moving the highlighted next object proximate to the target location; and User interaction with a highlighted part of the subassembly causes processing of the highlighted part based on the highlighted next object.

13. A virtual reality system according to claim 9 or any one of claims 9 to 12, wherein: The next object highlighted is the next part; The target location includes a highlighted part of the subassembly; and the user interacts with the highlighted part of the sub-assembly using the highlighted next object by moving the highlighted next object proximate to the target location; and User interaction with a highlighted part of the sub-assembly causes the highlighted next object to be added to the sub-assembly.

14. A virtual reality system according to claim 9 or any one of claims 9 to 13, wherein: The training space includes a manual for providing a user with access to an ordered list of steps for a set of instructions for a selected assembly.

15. A virtual reality system according to claim 9 or any one of claims 9 to 14, wherein: Visually highlighting the next object in the training space includes changing a color or brightness of the next object so that the next object is visually distinguished from other tools in the set of tools and from other components arranged in the working area.

16. A virtual reality system according to claim 9 or any one of claims 9 to 11, wherein: The processor circuit is further configured to: An instruction is received from a user to move in a training space such that a user's viewpoint in the training space is changed from a previous viewpoint to a new viewpoint, wherein displaying the training space by the processor circuit includes displaying the training space from the new viewpoint of the training space.

17. An augmented reality system for assisting a user in assembling a device, wherein: The augmented reality system comprises: a display configured to display a view of a user's environment including a graphical overlay; a processor circuit configured to identify the locations and contours of the following items in a work area of ​​the environment: a work surface, a set of tools disposed in the work area, and components disposed in the work area; a hand sensor configured to detect a position of a user's hand; A memory configured to store: A collection of assemblies, where: Each assembly in the set of assemblies is formed by combining parts of the assembly; and The parts of the assembly are a subset of the components arranged in the work area; A set of instructions, where: Each assembly in the set of assemblies is associated with a set of instructions from the set of instructions; Each of the sets of instructions includes an ordered list of steps that are executed to assemble an associated assembly by combining parts of the assembly; Each step in the ordered list of steps: creating a sub-assembly of the assembly by at least one of: adding at least one of the parts to the sub-assembly, or processing at least one of the parts of the assembly using at least one tool from the set of tools; identifying at least one of: a next part in the parts of the assembly of the subassembly, a location for placement of the next part, or a next tool in the set of tools for processing at least one of the parts of the assembly; Wherein, the processor circuit is further configured to: mapping the position and contour of the workbench, the set of tools, and the parts within the work area; receiving a selection of an assembly from the set of assemblies; receiving an instruction set for the selected assembly from the instruction set; sequentially causing a sequential list of steps of the received instruction set to be displayed on the graphical overlay, comprising: Display the current step in the ordered list of steps by: visually highlighting a next object in the graphical overlay, wherein the next object is a next part or a next tool; detecting user interaction with the highlighted next object using the hand sensor; and Upon detecting a user interaction with the highlighted next object, highlighting in the graphical overlay a target location for placing or interacting with the highlighted next object, wherein the target location comprises a location in the work area, a tool in the set of tools, or a part of the assembly; and When the user completes the current step, the next step is repeatedly selected as the current step in the ordered step list, and the current step in the ordered step list is displayed until all steps in the instruction set are completed, thereby assembling the associated assembly parts.

18. The augmented reality system of claim 17, wherein: The next object highlighted is the tool; The target location includes a highlighted part of the subassembly; and a user interacting with a highlighted part of the subassembly using the highlighted next object by moving the highlighted next object proximate to the target location; and User interaction with a highlighted part of the subassembly causes processing of the highlighted part based on the highlighted next object.

19. The augmented reality system according to claim 17 or 18, wherein: The next object highlighted is the next part; The target location includes a highlighted part of the subassembly; and the user interacts with the highlighted part of the sub-assembly using the highlighted next object by moving the highlighted next object proximate to the target location; and User interaction with a highlighted part of the sub-assembly causes the highlighted next object to be added to the sub-assembly.

20. The augmented reality system according to claim 17 or any one of claims 17-19, wherein: Visually highlighting the next object in the graphic overlay includes changing a color or brightness of the next object so that the next object is visually distinguished from other tools in the set of tools and from other components arranged in the work area.

Images