360° assistance for QCS scanners using mixed reality and machine learning technology

Abstract

An apparatus, method, and non-transitory machine-readable medium for providing 360° assistance to a quality control scanner using mixed reality and machine learning technologies, using an optical sensor, a display, a chatbot, a cloud service, and a processor operatively connected to the optical sensor and the display, wherein the processor identifies and detects problems with a field device using the optical sensor, guides a user to the location of the field device and to the scanner using the display, receives diagnostic information from a server associated with a field device in an industrial process control automation system to perform procedures or actions to resolve the problem, and further connects to a cloud server to obtain installation, commissioning, and annual maintenance contracts, as well as training modules for the quality control system.

Description

[Technical field]

[0001] The present disclosure relates generally to autonomous industrial plants, and more particularly to a system and method for 360° assistance of quality control system (QCS) scanners with mixed reality (MR) and machine learning techniques. Summary of the Invention [Problem to be solved by the invention]

[0002] Installing, upgrading and maintaining QCS scanners requires expert domain knowledge as it involves precise work with a series of operating procedures that involve radioactive sources and cannot be made to error. The industry people with expertise in QCS scanners are aging and retiring. New TACs and service areas are facing challenges in supporting QCS scanners due to capability gaps and lack of expertise. Troubleshooting QCS scanners requires domain knowledge to accurately pinpoint the potential problem and resolve this potential problem. Also, the time it takes to correct the problem may vary depending on the domain expertise. Training on QCS scanners requires time with the physical scanner and the physical environment for training.

[0003] The present disclosure provides systems and methods for 360° assistance of QCS scanners with mixed reality (MR) and machine learning techniques.

[0004] In a first embodiment, the device provides 360° assistance of the QCS scanner through mixed reality (MR) and machine learning techniques. The device includes an optical sensor, a display, and a processor operatively connected to the optical sensor and the display. The processor receives diagnostic information from a server associated with a field device of an industrial process control automation system, identifies a problem with the field device based on the diagnostic information, uses the optical sensor to find a field device corresponding to the identified problem, uses the display to guide a user to the location of the field device and the scanner unit associated with the problem, and further uses the display to provide instructions or actions required to resolve the problem.

[0005] In a second embodiment, a method provides 360° assistance for a QCS scanner through mixed reality (MR) and machine learning techniques. The method includes the steps of receiving diagnostic information from a server associated with a field device of an industrial process control automation system, identifying a problem with the field device based on the diagnostic information, detecting a field device corresponding to the identified problem using an optical sensor, guiding a user to a location of the field device associated with the problem and a scanner unit using a display, and further providing instructions or actions required to resolve the problem using the display.

[0006] In a third embodiment, the non-transitory medium provides 360° assistance of the QCS scanner through mixed reality (MR) and machine learning techniques. The instructions cause the one or more processors to receive diagnostic information from a server associated with a field device of an industrial process control automation system, identify a problem with the field device based on the diagnostic information, use an optical sensor to detect a field device corresponding to the identified problem, use a display to guide a user to the location of the field device associated with the problem and to the scanner unit, and further use the display to provide instructions or actions required to resolve the problem.

[0007] Other technical features will be readily apparent to one skilled in the art from the following figures, descriptions, and claims. [Brief description of the drawings]

[0008] For a more complete understanding of the present disclosure, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which:

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example industrial process control automation system according to the present disclosure.

[0010] [Diagram 2] FIG. 2 is a diagram illustrating an example apparatus for 360° assistance of a QCS scanner using mixed reality (MR) and machine learning techniques according to the present disclosure.

[0011] [Diagram 3] FIG. 3 is a diagram illustrating an example QCS scanner system for 360° assistance of the QCS scanner using mixed reality (MR) and machine learning techniques according to the present disclosure.

[0012] [Figure 4] FIG. 4 illustrates an example of a QCS scanner troubleshooting technique using augmented reality, chatbot, and machine learning techniques according to an embodiment of the present disclosure.

[0013] [Diagram 5] FIG. 5 illustrates an example QCS scanner training with augmented reality and chatbot technology according to an embodiment of the present disclosure.

[0014] [Figure 6] FIG. 6 illustrates the installation, commissioning and AMC of an example QCS scanner with augmented reality and chatbot technology according to an embodiment of the present disclosure.

[0015] [Figure 7A]FIG. 7A is an example flow diagram of troubleshooting a QCS scanner issue using augmented reality, chatbot, and machine learning techniques according to an embodiment of the present disclosure. [Figure 7B] FIG. 7B is an example flow diagram of troubleshooting a QCS scanner issue using augmented reality, chatbot and machine learning techniques according to an embodiment of the present disclosure.

[0016] [Figure 8A] FIG. 8A is an example flow diagram for installation and commissioning according to an embodiment of the present disclosure. [Figure 8B] FIG. 8B is an example flow diagram for installation and commissioning according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] 1-8B discussed below, and the various embodiments used to explain the principles of the present disclosure in this patent document, are for illustrative purposes only and should not be construed as limiting the scope of the invention in any manner. Those skilled in the art will appreciate that the principles of the present disclosure may be implemented in any type of suitably arranged device or system.

[0018] FIG. 1 illustrates an example industrial process control automation system 100 according to the present disclosure. As shown in FIG. 1, the system 100 includes various components that facilitate the production or processing of at least one product or other material. For example, the system 100 may be used to facilitate the control of components within one or more industrial plants. Each plant may represent one or more process facilities (or one or more portions thereof), such as one or more manufacturing facilities, for producing at least one product or other material. Generally, each plant may perform one or more industrial processes and may be referred to individually or collectively as a process system. A process system generally refers to any system or portion thereof that is configured to process one or more products or other materials in some manner.

[0019] In FIG. 1, the system 100 includes one or more sensors 102a and one or more actuators 102b. The sensors 102a and actuators 102b represent components within a process system that may perform any of a wide variety of functions. For example, the sensors 102a may measure a wide variety of characteristics within the process system, such as pressure, temperature, flow rate, basis weight, moisture, ash content, caliper, etc. Additionally, the actuators 102b may alter a wide variety of characteristics within the process system. Each of the sensors 102a includes any suitable structure for measuring one or more characteristics within the process system. Each of the actuators 102b includes any suitable structure for operating on or affecting one or more conditions within the process system.

[0020] At least one network 104 is coupled to the sensors 102a and the actuators 102b. The network 104 facilitates interaction with the sensors 102a and the actuators 102b. For example, the network 104 can transmit measurement data from the sensors 102a and provide control signals to the actuators 102b. The network 104 can represent any suitable network or combination of networks. As a specific example, the network 104 can represent at least one Ethernet network, an electrical signal network (such as a HART or FOUNDATION FIELDBUS network), a pneumatic control signal network, or any other or new type(s) of network.

[0021] The system 100 also includes various controllers 106. The controllers 106 may be used within the system 100 to perform various functions to control one or more industrial processes. For example, a first set of controllers 106 may use measurements from one or more sensors 102a to control the operation of one or more actuators 102b. A second set of controllers 106 may be used to optimize control logic or other operations performed by the first set of controllers. A third set of controllers 106 may be used to perform new functions.

[0022] Controllers 106 are often arranged hierarchically within a system. For example, various controllers 106 may be used to control individual actuators, collections of actuators forming machines, collections of machines forming units, collections of units forming plants, and collections of plants forming an enterprise. A particular example of a hierarchical arrangement of controllers 106 is defined as the "Purdue" model of process control. Controllers 106 at various hierarchical levels may communicate via one or more networks 108 and associated switches, firewalls, and other components.

[0023] Each controller 106 includes any suitable structure for controlling one or more aspects of an industrial process. At least some of the controllers 106 may represent multivariable controllers, such as, for example, proportional-integral-derivative (PID) controllers or other types of controllers implementing model predictive control or other advanced predictive control. As a particular example, each controller 106 may represent a computing device running a real-time operating system, a WINDOWS operating system, or other operating system.

[0024] Operator access and interaction with the controllers 106 and other components of the system 100 can be provided through various operator consoles 110. Each operator console 110 can be used to provide information to and receive information from an operator. For example, each operator console 110 can provide information to an operator identifying a current state of the industrial process, such as the values ​​of various process variables and warnings, alarms, or other conditions associated with the industrial process. Additionally, each operator console 110 can receive information that affects how the industrial process is controlled, such as by receiving set points or control modes for the process variables controlled by the controller 106, or other information that changes or affects how the controller 106 controls the industrial process.

[0025] Multiple operator consoles 110 may be grouped together and used in one or more control rooms 112. Each control room 112 may include any number of operator consoles 110 in any suitable arrangement. In some embodiments, multiple control rooms 112 may be used to control an industrial plant, such as where each control room 112 contains an operator console 110 used to manage a separate part of the industrial plant.

[0026] Each operator console 110 includes any suitable structure for displaying information to and interacting with an operator. For example, each operator console 110 can include one or more processing devices 114, such as one or more processors, microprocessors, microcontrollers, field programmable gate arrays, application specific integrated circuits, discrete logic devices, or other processing or control devices. Additionally, each operator console 110 can include one or more memories 116 that store instructions and data used, generated, or collected by the processing device(s) 114. Each operator console 110 can also include one or more network interfaces 118, such as one or more Ethernet interfaces or wireless transceivers, that facilitate communication over at least one wired or wireless network.

[0027] According to the present disclosure, a technique is provided for 360° assistance of a QCS scanner through mixed reality (MR) and machine learning techniques, and one or more components of the system 100 (e.g., the operator console 110) can be configured to perform one or more operations related to this technique.

[0028] Although Fig. 1 illustrates an example of an industrial process control automation system 100, various modifications may be made to Fig. 1. For example, industrial control and automation systems come in a wide variety of configurations. The system 100 illustrated in Fig. 1 is intended to illustrate one example of an operating environment in which a pressure sensor may be used.

[0029] FIG. 2 illustrates an example apparatus for 360° assistance of a QCS scanner with mixed reality (MR) and machine learning techniques according to the present disclosure. Specifically, FIG. 2 illustrates an example computing device 200. In some embodiments, the computing device 200 may represent an operator station, a server, a remote server or device, or a mobile device. The computing device 200 may be used to execute applications. For ease of explanation, the computing device 200 is described as being used in the system 100 of FIG. 1, but the apparatus may be used in any other suitable system, whether or not related to industrial process control automation.

[0030] 2, computing device 200 includes at least one processor 202, at least one storage device 204, at least one communication unit 206, and at least one input / output (I / O) unit 208. Each processor 202 is capable of executing instructions, such as instructions that may be loaded into memory 210. Each processor 202 may be any suitable processing device, such as one or more microprocessors, microcontrollers, digital signal processors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or discrete circuits.

[0031] Memory 210 and persistent storage 212 are examples of storage device 204, which represents any structure or structures configured to store and facilitate retrieval of information, such as data, program code, and / or other suitable information, either temporarily or permanently. Memory 210 may represent random access memory or any other suitable volatile or non-volatile storage device or devices. Persistent storage 212 may include one or more components or devices that support long-term retention of data, such as read-only memory, a hard drive, flash memory, or an optical disk.

[0032] The communications unit 206 supports communication with other systems or devices. For example, the communications unit 206 may include at least one network interface card or a wireless transceiver that facilitates communication over at least one wired or wireless network. The communications unit 206 may support communication over any suitable physical or wireless communication link(s).

[0033] The I / O unit 208 allows for the input and output of data. For example, the I / O unit 208 may provide a connection for user input through a keyboard, mouse, keypad, touch screen, gesture control, image processing, or other suitable input device. Additionally, the I / O unit 208 may send output to a display, printer, or other suitable output device.

[0034] Figure 3 illustrates an example QCS scanner system 300 for 360° assistance of a QCS scanner 310 with mixed reality (MR) and machine learning techniques according to the present disclosure. The embodiment of the example QCS scanner system 300 illustrated in Figure 3 is for illustrative purposes only. Figure 3 is not intended to limit the scope of the present disclosure to any particular implementation.

[0035] The QCS scanner system 300 offers mixed reality (MR) (augmented reality (AR) / virtual reality (VR), machine learning, and chatbot solutions to resolve potential issues. Using MR, commissioning of the QCS system 300 becomes safer, easier, and more user-friendly by augmenting the physical conditions with interactive guidance for the installation and upgrade of the QCS scanner.

[0036] The QCS system 300 integrates diagnostic messages and fault information from the QCS scanner with the HoloLens 305. The QCS system 300 receives solutions from local / centralized solution centers that enable interactive chatbots and machine learning for troubleshooting.

[0037] The QCS system creates virtual training for the QCS scanner in VR and AR, which reduces the overall cost of training and physical hardware. The QCS system provides instructions on safe handling of radioactive sources in hazardous environments. The QCS system creates mimics of scanner components, which provide detailed information on wiring details, equipment location identification, checkpoints, etc.

[0038] The AR solution provides an augmented physical scanner with real-time data for troubleshooting. The AR solution enhances the step-by-step instructions for installing QCS scanners. The AR solution can upload real-time scanner status.

[0039] Machine learning and chatbot 315 provides easy troubleshooting solutions based on historical data through interactive live chat sessions between the machine and expert chatter. Machine learning and chatbot 315 can record the problem and its resolution steps for future use.

[0040] The VR solution can provide an alternative to practice the installation and commissioning of the QCS scanner without having access to expensive physical components. The VR solution can simulate QCS scanner scenarios, including real-life system training for troubleshooting, and display the live status and hints of the QCS system.

[0041] The phrase "360° Support" for QCS Scanner refers to the holistic support of QCS Scanner. There are four main modules of QCS Scanner regarding support: Module 1: Troubleshooting QCS Scanner Issues During Processing; Module 2: Training; Module 3: Installation and Commissioning of QCS Scanner; and Module 4: Annual Maintenance / Regular Checks. All the modules mentioned for QCS Scanner Support require different technologies / technology combinations and approaches to achieve standardized, time-bound, predictable and reliable in the process.

[0042] FIGURE 4 illustrates an example QCS scanner troubleshooting technique 400 using augmented reality, chatbot, and machine learning techniques according to embodiments of the present disclosure. The embodiment of the example QCS scanner troubleshooting technique 400 illustrated in FIGURE 4 is for illustrative purposes only and is not intended to limit the scope of the present disclosure to any particular implementation.

[0043] The QCS troubleshooting methodology 400 includes HoloLens 405, chatbot 410, problem identification 415, eDocumentation 420, machine learning server 425, and expert support 430. HoloLens 405 is a holographic computer built to identify the QCS scanner and its internal parts, virtual wiring layout, connection identification information, scanner mechanical part identification information, etc. based on the scanner version that also allows video streaming of the scanner for remote assistance.

[0044] The chatbot 410 can provide interactive voice-based chatbot technology that accepts voice input from a user and provides the necessary output to guide the user to perform a required action.

[0045] Problem identification 415 involves integrating the HoloLens with the QCS server and the QCS scanner to provide scanner-related diagnostics based on the diagnostic information. The HoloLens can guide the user to the problem location or scanner part and provide steps or actions to take to resolve the problem.

[0046] eDocumentation 420 enables HoloLens to identify objects and provide information about the objects, such as wiring diagrams, machine connections, test points, etc. eDocumentation also enables users to receive documentation they require with the aid of chatbots, reducing search time and data availability, improving the user experience. HoloLens can identify objects related to field equipment. Users can provide commands that HoloLens receives using audio sensors or from external devices. HoloLens displays documentation corresponding to the document type of the identified object.

[0047] The machine learning server 425 is a cloud service provided to solve problems based on the severity of the problem. A user can connect to the machine learning server with a chatbot, request a solution, and it can provide a solution based on similar problems that have occurred in the past, and it can also record the procedural steps taken to solve the current problem, allowing the system to provide a more reliable and accurate solution in the future.

[0048] Expert support 430 is used when the machine learning server is unable to resolve the issue and the user requires expert assistance. The HoloLens can request the machine learning server to connect to an available expert. Once connected to the expert, the expert can explain the problem while demonstrating how to solve it. Once the issue is resolved, the machine learning server can record the steps taken to resolve the issue.

[0049] FIGURE 5 illustrates an example QCS scanner training system 500 with augmented reality and chatbot technology according to an embodiment of the present disclosure. The exemplary QCS scanner training 500 embodiment illustrated in FIGURE 5 is for illustrative purposes only and is not intended to limit the scope of the present disclosure to any particular implementation.

[0050] QCS Training 500 includes HoloLens 505 with Chatbot 510, cloud-based training manual with cloud-based eDocument & Video Training 515 and cloud-based Virtual Training Module 520, as well as Virtual Training 525.

[0051] The wearable or HoloLens 505 is a holographic computer that can simulate a QCS scanner in a virtual world that can image a virtual QCS scanner. The HoloLens 505 can show what a physical scanner would look like and show internal parts that can be virtually imaged to help a user learn about the various components (sensors, mechanisms, hardware and software configurations, etc.) before running the actual scanner.

[0052] The chatbot 510 is an interactive, voice-based chatbot capable of accepting voice input from a user and providing the necessary output to guide the user to take the necessary action to resolve the problem.

[0053] Cloud-based eDocument & Video Training 515 can cover basic introduction and industrial use of QCS scanner. Cloud-based Virtual Training Module 520 can cover knowledge on QCS scanner, sensors, mechanism, hardware and software configuration, operation, service and troubleshooting. Examples of training modules can be Installation & Commissioning Module 530, QCS Application Module 535, Troubleshooting Module 540, AMC & Service Module 545, etc.

[0054] Using virtual training 525, a user wears a HoloLens and connects to a cloud-based training module. The HoloLens selects a persona for the training module. Using virtual reality, chatbot users can commission, troubleshoot, and view usage of plant scenarios without using or having access to a physical scanner.

[0055] FIG 6 illustrates an example installation, commissioning and AMC system 600 of a QCS scanner with augmented reality and chatbot technology according to an embodiment of the present disclosure. The embodiment of the example system 600 of a QCS scanner illustrated in FIG 6 is for illustration purposes only. FIG 6 is not intended to limit the scope of the present disclosure to any particular implementation.

[0056] The QCS scanner system 600 includes installation, commissioning, and annual maintenance contract (AMC). The system 600 includes a HoloLens 605 with chatbot 610, a cloud-based installation & AMC module 615, installation and commissioning 620, expert support 625, and AMC activities 630.

[0057] The HoloLens 605 is a holographic computer created to identify site locations and provide prerequisites and checks for installing the QCS scanner. Using augmented reality, the HoloLens 605 can guide users during the installation and commissioning of the QCS scanner, various sensors, as well as internal components. With the HoloLens, remote assistance is available to receive expert advice.

[0058] The chatbot 610 is an interactive, voice-based chatbot that is capable of accepting voice input from a user and providing the necessary output to guide the user to take the necessary action to resolve the problem.

[0059] Cloud based installation & AMC module 615 includes various modules that can be accessed based on requirements from the cloud that are broadly categorized into sub-modules, including an installation module 635 to support various versions of QCS scanners, a QCS sensor module 640, a QCS software installation and configuration module 645, AMC activities for QCS scanner module 650, etc.

[0060] Installation and Commissioning 620 includes the HoloLens connecting to a cloud service. The HoloLens selects an available QCS scanner version and sensor for commissioning. Installation and Commissioning 620 can guide the HoloLens through a step-by-step procedure for commissioning the QCS scanner along with software installation and configuration to fully launch the QCS scanner.

[0061] Expert Support 625 can provide expert assistance with installation, commissioning, and AMC in HoloLens if the user does not understand some part of the process or an issue with the QCS scanner. When the HoloLens is talking to the user, it can connect with an expert and provide a view of the QCS scanner. The expert can control the HoloLens and show the components to the user. In this way, the expert can better explain the steps or procedures to resolve the issue.

[0062] The AMC activity 630 is based on a customer record system that allows the creation of AMC checklists. The AMC activity 630 can guide the user to control the HoloLens to perform ACM activities that allow the technician to collect reports on the running activities and comments. The HoloLens can generate a final report of the AMC activities on the customer and user records.

[0063] 7A and 7B illustrate example methods 700, 701 for troubleshooting QCS scanner issues using augmented reality, chatbot, and machine learning techniques according to embodiments of the present disclosure. For example, the methods described in Figures 7A and 7B can be performed in conjunction with computing device 200 of Figure 2.

[0064] In operation 705, the computing device 200 may detect a problem with the QCS scanner. In some embodiments, detecting the problem by the computing device 200 includes receiving diagnostic information from a server associated with a field device, such as a QCS scanner, in an industrial process control automation system. The problem with the field device includes a malfunction that causes the field device not to function to proper operational requirements.

[0065] In operation 710, the computing device 200 can connect to the QCS server and receive a list of possible problems. The list of possible problems can include typical problems identified with the particular machine itself or from a list of commonly used malfunctions of the equipment type.

[0066] In operation 715, the computing device 200 may receive a voice command to select a potential problem from a list of potential problems. The list of potential problems may be displayed on a display or provided to a user as an audio output. In some embodiments, the computing device 200 may identify a problem with the field device based on the diagnostic information.

[0067] In operation 720, the computing device 200 can capture a QCS scanner using an optical sensor on the HoloLens. The computing device 200 can detect a field device that corresponds to the problem identified by the optical sensor. Once the field device is captured and detected, the computing device can identify the particular component that corresponds to the identified problem.

[0068] In operation 725, the computing device 200 can display instructions on the display of the HoloLens that correspond to components related to the possible problem with the QCS scanner. The computing device 200 can use the display to guide the user to the location of the field equipment and scanner portion involved in the problem.

[0069] In operation 730, the computing device 200 may receive a request via voice command for relevant documentation and procedures to resolve the problem. The computing device 200 may provide the procedures or actions necessary to resolve the problem. The necessary procedures or actions may be displayed on the display. Particular components related to steps of the procedure may be highlighted or recorded on the display along with the relevant documentation. The computing device 200 may display the relevant documentation on the display in a manner that separates the highlighted or recorded components.

[0070] In operation 735, the computing device 200 may determine whether the problem has been resolved. If the problem has been resolved, the computing device 200 proceeds to operation 780. If the problem has not been resolved, the computing device 200 proceeds to operation 740.

[0071] In operation 740, the computing device 200 may detect that the problem has not been resolved. The computing device 200 may receive operational data from the QCS server and determine that the field device is still not operating efficiently.

[0072] In operation 745, the computing device 200 can connect the HoloLens to a cloud-based service on a machine learning server for additional assistance with the anticipated problem. The cloud-based service can be directly associated with a particular component.

[0073] In operation 750, the computing device 200 can use the chatbot service to provide detailed information about the problem to the machine learning server. The detailed information can include information of the field equipment from the QCS server along with information captured by the computing device 200. Information captured at the computing device can include live feeds or frames captured from optical sensors, audio captured from a user, frames captured from a display of the process used to fix the problem, etc.

[0074] In operation 755, the computing device 200 may receive a solution associated with the problem based on past history from the machine learning server. The past history may include problems previously solved from the field device and other field devices of the same type. The machine learning server may provide an optimal solution based on all the input data or various alternative options.

[0075] In operation 760, the computing device 200 may determine whether the problem has been resolved. If the problem has been resolved, the computing device 200 proceeds to operation 780. If the problem has not been resolved, the computing device 200 proceeds to operation 765.

[0076] In operation 765, the computing device 200 may request the machine learning server to connect to a subject matter expert at a technical assistance center (TAC) center. A subject matter expert may be someone who has experience using a particular type of field device or dealing with a particular problem.

[0077] In operation 770, the computing device 200 can provide detailed information about the problem and a live feed from the optical sensor of the HoloLens to the subject matter expert, who can connect live or send relevant information related to the field equipment.

[0078] In operation 775, the computing device 200 may receive an expert report from a subject matter expert that resolves the problem. The expert report may include step-by-step instructions for resolving the problem. Each step may also include highlighting or identifying markers on the display for various components that correspond to the respective step.

[0079] In operation 780, the computing device 200 may record the problem type and the resolution procedure in the machine learning server. The problem type and the resolution procedure may be related to a particular component or collection of components, or may be related to a malfunction of a field device. In operation 785, the computing device 200 may determine that the problem has been resolved.

[0080] Although Figures 7A and 7B illustrate an example of a 360° assistance method 700, 701 for a QCS scanner using mixed reality (MR) and machine learning techniques, various modifications may be made to Figure 7. For example, the various steps illustrated in Figure 7 may overlap, occur in parallel, occur in a different order, or occur any number of times.

[0081] 8A and 8B show an example flow diagram for installation and commissioning according to an embodiment of the present disclosure. For example, the method described in FIG. 8A and 8B can be performed in conjunction with the installation, commissioning and AMC system 600 of FIG.

[0082] In operation 805, the installation, commissioning and AMC system 600 may begin the installation, commissioning and AMC system of the QCS scanner.

[0083] In operation 810, the installation, commissioning and AMC system 600 can connect to a cloud server to obtain a persona for installation, commissioning and AMC from the wearable device.

[0084] In operation 815, the installation, commissioning and AMC system 600 may determine whether the installation and commissioning persona is selected.

[0085] In operation 820, the installation, commissioning and AMC system 600 can use voice commands for the wearable device to request installation modules required for the supported version of the QCS scanner from the cloud server.

[0086] In operation 825, the installation, commissioning and AMC system 600 can obtain a list of supported hardware and software installations of QCS scanners and sensors.

[0087] In operation 830, the installation, commissioning and AMC system 600 can use voice commands directed to the wearable to request required modules from a cloud server, including a QCS scanner installation module, a QCS sensor module, and a QCS software installation configuration module.

[0088] In operation 835, the installation, commissioning and AMC system 600 may receive from the cloud server a set of procedures that need to be performed to install and commission the QCS scanner on the wearable device.

[0089] In operation 840, the installation, commissioning and AMC system 600 can perform the installation and commissioning from the instructions and evaluate the results on the wearable device.

[0090] In operation 845, the installation, commissioning and AMC system 600 can determine whether the user is able to execute the instruction or set of instructions.

[0091] In operation 850, the installation, commissioning and AMC system 600 can receive assistance from subject matter experts to resolve issues via interactive chat over live video.

[0092] In operation 855, the installation, commissioning and AMC system 600 may select an AMC module.

[0093] In operation 860, the installation, commissioning and AMC system 600 can use voice commands for the wearable device to request AMC records and checklists from the cloud server based on the version of the QCS scanner.

[0094] In operation 865, the installation, commissioning and AMC system 600 may receive a checklist of selected QCS scanners for AMC.

[0095] In operation 870, the installation, commissioning and AMC system 600 may receive from the cloud server a set of procedures necessary to execute for the AMC QCS scanner in the wearable device.

[0096] In operation 875, the installation, commissioning and AMC system 600 can perform AMC from the instructions and evaluate the results on the wearable device.

[0097] In operation 880, the installation, commissioning and AMC system 600 can determine whether the user is able to execute the command or set of commands.

[0098] In operation 885, the installation, commissioning and AMC system 600 can receive assistance from subject matter experts to resolve issues via interactive chat over live video.

[0099] While Figures 8A and 8B illustrate an example of a 360° assistance method 800, 801 for a QCS scanner using mixed reality (MR) and machine learning techniques, various modifications may be made to Figure 8. For example, the various steps illustrated in Figure 8 may overlap, occur in parallel, occur in a different order, or occur any number of times.

[0100] It may be helpful to provide definitions of certain words and phrases used throughout this patent document. The words "transmit," "receive," and "communicate," and their derivatives, include both direct and indirect communication. The terms "include" and "comprise," and their derivatives, mean an inclusion without limitation. The term "or" is inclusive and means "and / or." The phrase "associated with," and its derivatives, may mean including, contained within, interconnected, containing, being contained within, interconnected, coupled with, being able to communicate, coordinating, interleaving, juxtaposing, being adjacent to, being linked with, having, having an attribute, having a relationship with, and the like. The phrase "at least one of," when used with a list of items, means that various combinations of one or more of the listed items may be used and that only one item in the list may be required. For example, "at least one of A, B, and C" includes any of the following combinations: A, B, C, A and B, A and C, B and C, and A and B and C.

[0101] Although the present disclosure has described certain embodiments and generally associated methods, modifications and variations of these embodiments and methods will be apparent to those skilled in the art. Thus, the above description of exemplary embodiments does not define or constrain the present disclosure. Other changes, substitutions, and alterations are also possible without departing from the spirit and scope of the present disclosure, as defined by the appended claims.

Claims

1. A method (700, 701) of using a chatbot application for interactive communication between a user and a system, comprising: receiving (710) diagnostic information indicative of a malfunction of a field device (310) from a server (425) communicatively coupled to the field device (310) of the industrial process control automation system (100); identifying a problem with the field device based on the diagnostic information (715); Detecting (720) the field device corresponding to the identified problem using an optical sensor (102a); displaying (725) to the user the location of the field device involved in the identified problem using a display (305); using said display to display (730) the steps or actions required to resolve said identified problems; Connecting to the user's device via a cloud server (430) to obtain Quality Control System (QCS) scanner installation, commissioning and annual maintenance contract (AMC) and Quality Control System (QCS) training modules (530-545); providing a solution for troubleshooting obtained from a machine learning server based on past data of the field device to the user using the chatbot application; A method (700, 701).

2. A method (700, 701) of using a chatbot application for interactive communication between a user and a system, comprising: receiving (710) diagnostic information indicative of a malfunction of a field device (310) from a server (425) communicatively coupled to the field device (310) of the industrial process control automation system (100); identifying a problem with the field device based on the diagnostic information (715); Detecting (720) the field device corresponding to the identified problem using an optical sensor (102a); displaying (725) to the user the location of the field device involved in the identified problem using a display (305); using said display to display (730) the steps or actions required to resolve said identified problems; If the company determines that the necessary steps or actions have not resolved the identified problem, providing (750) details of the identified problem to a cloud-based service on a machine learning server; receiving 755 from the machine learning server a related solution based on past history of similar field devices, the related solution being associated with installation, commissioning, and annual maintenance contracts (AMCs) for quality control system (QCS) scanners, and training for the quality control system (QCS); providing the received related solutions to the user utilizing the chatbot application (700, 701).

3. An apparatus (200), a processor operatively connected to the optical sensor (102a) and the display (305), the processor comprising: receiving (710) diagnostic information indicative of a malfunction of a field device (310) from a server (425) communicatively coupled to the field device (310) of the industrial process control automation system (100); identifying a problem with the field device based on the diagnostic information (715); detecting (720) the field device corresponding to the identified problem using the optical sensor (102a); using the display to indicate to a user the location of the field device involved in the identified problem (725); using said display to display (730) the steps or actions required to resolve said identified problems; Connecting to the user's device via a cloud server (430) to obtain Quality Control System (QCS) scanner installation, commissioning and annual maintenance contract (AMC) and Quality Control System (QCS) training modules (530-545); providing a solution for dealing with a problem acquired from the machine learning server based on past data of the field device to the user using a chatbot application; An apparatus (200) configured to execute the steps of:

Images