Systems, methods, and apparatuses for improving performance of executing workflow operations

Abstract

Systems, methods, and devices for improving performance of performing workflow operations. A voice-controlled device for performing workflow operations is described. The voice-controlled device can include a microphone, a speaker, and a processor. In some examples, the voice-controlled device can generate, via the speaker, a voice prompt associated with a task of a workflow, and recognize, via the microphone, a voice response received from a worker. In this regard, the voice prompt and the voice response can be part of a voice dialog. Further, the processor of the voice-controlled device can recognize a performance state associated with the performance of the task prior to providing a next voice prompt after the voice prompt. In this regard, the performance state can be recognized based on analyzing the voice dialog using a machine learning model. Further, the voice-controlled device can generate a message including a suggestion to improve the performance state of the task.

Description

TECHNICAL FIELD

[0001] The example embodiments described herein relate generally to systems, methods, and apparatuses for improving performance of executing workflow operations, and more particularly to providing suggestions to a worker to improve the worker's performance in executing a workflow operation. BACKGROUND

[0002] In many environments, such as but not limited to, distribution centers, warehouses, inventory, industrial sites, etc., certain activities or tasks are performed by workers in the form of a workflow, where each task is broken into a series or sequence of steps to be performed to complete the task. For example, a pick workflow can be related to operations for picking various items in a material handling site. In some examples, the tasks of the workflow are performed using a voice dialogue, where instructions for performing each step of the workflow are provided to the worker as a voice prompt (as in the case of an interactive voice response, IVR, system), to which the worker can respond in the form of a voice response (i.e., an audible or spoken response). In some examples, the performance of the workflow execution depends on various factors, such as but not limited to, the quality of performing the tasks, the turnaround time for performing the tasks, the seamless exchange of the voice prompts and voice responses, etc. Real-time monitoring of the performance of the workflow execution helps in various ways, such as but not limited to, efficiently planning the workflow operations for a work shift, planning the work schedule of the workers, assigning delivery deadlines to customers, etc. Generally, in cases where the workflow operations are performed using a voice dialogue, real-time monitoring of the performance of the execution of the workflow operations is challenging and has associated limitations. SUMMARY

[0003] Various example embodiments described herein relate to a voice control apparatus. The voice control apparatus can include a microphone, a speaker, and a processor. The processor can be communicatively coupled to at least one of the microphone and the speaker. The processor can be configured to generate, via the speaker, a voice prompt associated with a task of a workflow. Further, the processor can be configured to identify, via the microphone, a voice response from a worker in response to the voice prompt. In this regard, the voice prompt and the voice response can be part of a voice dialogue. Further, the processor of the voice control apparatus can be configured to identify a performance state associated with the execution of the task. The processor can identify the performance state prior to a next voice prompt after providing the voice prompt. The performance state can be identified based on analyzing the voice dialogue using a machine learning model. Further, the processor can be configured to generate a message including a suggestion to improve the performance state of the task.

[0004] According to some example embodiments, the performance state can be indicative of at least one of: a delay in performance of the task of the workflow; a coordination gap corresponding to the voice prompt and the voice response provided by the worker; and an incorrect performance of the task of the workflow.

[0005] According to some example embodiments, the processor of the voice control device can be configured to access a data file comprising a historical voice conversation corresponding to a past performance of the task. Further, the processor can be configured to parse the data file to identify a contextual keyword and a contextual parameter associated with the contextual keyword from a plurality of historical voice responses in the historical voice conversation. In some examples, the contextual parameter can comprise at least one of: a frequency of occurrence of the contextual keyword in the historical voice conversation, and timing information corresponding to each occurrence of the contextual keyword in the historical voice conversation. Further, the processor can be configured to provide the contextual keyword and the contextual parameter as input features to train the machine learning model.

[0006] According to some example embodiments, the processor can be further configured to identify a pattern associated with the contextual keyword and the contextual parameter using the machine learning model. In this regard, the machine learning model can identify the pattern based on pre-defined rules defined according to a type of task of the workflow. Further, the processor can be configured to predict a classification indicative of the performance state associated with the performance of the task by using the pattern.

[0007] According to some example embodiments, the suggestion to improve the performance state can be indicative of at least one of: (a) a correct location for picking up an item; (b) a correct destination for placing the item; (c) a route to reach one of: an item pick-up location or an item placement destination; (d) an expected voice response to the voice prompt according to the workflow; (e) retraining a speech recognition model according to a noise level in a work environment; (f) retraining the speech recognition model according to a speaker dependent voice template; (g) a new workflow that is preferred for the worker; and (h) an option to reassign the task of the worker to another worker.

[0008] According to some example embodiments, the task can be picking up an item. In this regard, the processor can be configured to identify the performance state indicative of one of: (a) slow pick-up, in a case where a time taken to pick up the item is slower than an expected time to pick up the item; and (b) normal pick-up, in a case where the time taken to pick up the item is consistent with the expected time to pick up the item.

[0009] Some example embodiments described herein relate to a system comprising a processor. The processor can be configured to receive a voice dialogue associated with a task of a workflow to be performed on a voice-controlled device. The voice dialogue can include a first voice prompt and a first voice response to the first voice prompt. Further, the processor can be configured to identify a performance state associated with performance of the task based on analyzing the voice dialogue using a machine learning model. In this regard, the performance state can be identified prior to a second voice prompt provided after the first voice prompt. Further, the processor can be configured to generate a message indicating a suggestion to improve the performance state of the task.

[0010] According to some example embodiments, the performance state can indicate at least one of: a delay in performance of the task of the workflow; a coordination gap corresponding to the voice prompt and the voice response provided by the worker; and an incorrect performance of the task of the workflow.

[0011] According to some example embodiments, the processor can be configured to access a data file comprising historical voice dialogues corresponding to past performances of the task. Further, the processor can be configured to parse the data file to identify a contextual keyword and a contextual parameter associated with the contextual keyword from a plurality of historical voice responses in the historical voice dialogues. In some examples, the contextual parameter can include at least one of: a frequency of occurrence of the contextual keyword in the historical voice dialogues, and timing information corresponding to each occurrence of the contextual keyword in the historical voice dialogues. Further, the processor can be configured to provide the contextual keyword and the contextual parameter as input features to train the machine learning model.

[0012] According to some example embodiments, the processor can be further configured to identify a pattern associated with the contextual keyword and the contextual parameter using the machine learning model. In this regard, the machine learning model can identify the pattern based on pre-defined rules defined according to a type of task of the workflow. Further, the processor can be configured to predict a classification indicating the performance state associated with the performance of the task by using the pattern.

[0013] According to some example embodiments, the suggestion to improve the performance state can indicate at least one of: (a) a correct location for picking up an item; (b) a correct destination for placing the item; (c) a route to reach one of: an item pick-up location or an item placement destination; (d) an expected voice response to the voice prompt according to the workflow; (e) retraining a speech recognition model according to a noise level in a work environment; (f) retraining the speech recognition model according to a speaker-dependent voice template; (g) a new workflow that is preferred for the worker; and (h) an option to reassign the task of the worker to another worker.

[0014] According to some example embodiments, the task can be picking up an item. In this regard, the processor of the system can be configured to identify the performance state indicating one of: (a) slow picking up, in a case where a time taken to pick up the item is slower than an expected time to pick up the item; and (b) normal picking up, in a case where the time taken to pick up the item is consistent with the expected time to pick up the item.

[0015] In some example embodiments, the processor of the system can be configured to transmit a file comprising the workflow on an electronic device. In this regard, the workflow can be executed based on an exchange of messages between the electronic device and the voice-controlled device. Further, the processor can be configured to receive workflow execution data comprising the voice conversation associated with the execution of a task of the workflow by the worker.

[0016] Some example embodiments described herein relate to a method for improving a performance state of a task of a workflow. The method includes receiving a voice conversation associated with the task of the workflow, the voice conversation comprising a first voice prompt and a first voice response to the first voice prompt. Further, the method includes identifying a performance state associated with the execution of the task based on analyzing the voice conversation using a machine learning model. In this regard, the performance state can be identified prior to a second voice prompt provided after the first voice prompt. Further, the method includes generating a message indicating a suggestion to improve the performance state of the task.

[0017] In some example embodiments, the method can further include identifying a pattern associated with the contextual keyword and the contextual parameter using the machine learning model. In this regard, the machine learning model can be used to identify the pattern based on predefined rules defined according to a type of task of the workflow. Further, the method can include predicting a classification indicating the performance state associated with the execution of the task by using the pattern.

[0018] According to example embodiments, the method can include transmitting, on the electronic device, a file comprising the workflow. In this regard, the workflow can be executed based on an exchange of messages between the electronic device and the voice-controlled device. Further, the method can include receiving workflow execution data, which can include the voice conversation associated with the worker's performance of the tasks of the workflow. BRIEF DESCRIPTION OF DRAWINGS

[0019] The accompanying drawings illustrate embodiments of the application and, together with the description (including the general description and the detailed description below), serve to explain the features of the application.

[0020] Figure 1 A schematic diagram of a workflow execution system according to example embodiments is shown;

[0021] Figure 2 A schematic block diagram of a workflow execution system according to example embodiments is shown;

[0022] Figure 3 A schematic block diagram of a workflow database according to example embodiments is shown;

[0023] Figure 4 A schematic block diagram of a workflow system for improving workflow execution according to example embodiments is shown;

[0024] Figure 5 An example voice-controlled device for performing workflow operations according to example embodiments is shown;

[0025] Figure 6 A block diagram of a voice-controlled device for performing workflow operations according to example embodiments is shown;

[0026] Figure 7 A schematic diagram of an example electronic device for performing workflow operations according to example embodiments is shown;

[0027] Figure 8 A schematic diagram of another example electronic device for performing workflow operations according to another example embodiment is shown;

[0028] Figure 9 A flowchart representing a method for improving performance in performing workflow operations according to example embodiments is shown;

[0029] Figure 10 A flowchart representing a method for identifying a performance state associated with performance of a workflow operation according to example embodiments is shown;

[0030] Figure 11A flowchart representing a method for providing suggestions to improve worker performance in performing workflow operations is shown in accordance with an example embodiment; and

[0031] Figure 12 An example scenario depicting a workflow operation performed by an operator is shown in accordance with an example embodiment. DETAILED DESCRIPTION

[0032] The present application now will be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all embodiments of the applications are shown. Indeed, these applications can be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will satisfy applicable legal requirements. Like numbers refer to like elements throughout. As used herein, terms such as "front," "rear," "top," "bottom," "outer," "inner," and the like refer to the examples provided below for illustrative purposes to describe the relative position of certain devices or portions of devices. The terms used in this patent are not meant to be limiting, and the devices described herein or portions thereof can be attached or utilized in other orientations.

[0033] The term "comprising" means including, but not limited to, and should be interpreted in the manner set out in the passage of the Patent Act relating to definition of expression "comprising". It is understood that the use of broadening terms such as "comprising", "including", and "having" provide support for narrow terms such as "consisting of", "consisting essentially of", and "consisting as of".

[0034] The phrases "in one embodiment", "according to one embodiment", and the like, generally mean that a particular feature, structure, or characteristic described in connection with the phrase is included in at least one embodiment of the application, and can be included in more than one embodiment of the application (importantly, such phrases are not necessarily referring to the same embodiment).

[0035] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations.

[0036] The term "about" or "approximately," and the like, when used in connection with a number, can mean close to that number, or alternatively, within a range of that number as understood by persons of ordinary skill in the art.

[0037] If the specification states a component or feature "may," "could," "would," "should," "can," "will," "preferably," "possibly," "typically," "options," "for example," "often," or "might" (or other such term) be included or have a particular property, that particular component or feature is not required to be included or to have the particular property.

[0038] As used herein, the term "transmitter" refers to any component that can generate radio waves for communication purposes, while "receiver" is used generically to refer to any component that can receive radio waves and convert that information into a usable form. "Transceiver" refers generically to a component that can both generate radio waves and receive radio waves and thus would be contemplated when discussing either a transmitter or a receiver.

[0039] The term "processor" is used herein to refer to any programmable microprocessor, microcomputer, or one or more multiple processor chips that can be configured by software instructions (applications) to perform a variety of functions, including the functions of the various implementations described above. In some devices, multiple processors can be provided, such as one processor dedicated to wireless communication functions and one processor dedicated to running other applications. Software applications can be stored in internal memory before being accessed and loaded into the processor. The processor can include internal memory sufficient to store the application software instructions. In many devices, the internal memory can be a volatile or nonvolatile memory, such as flash memory, or a mix of both. Memory can also be internal to another computing resource (e.g., enabling computer-readable instructions to be downloaded over the Internet or another wired or wireless connection).

[0040] For the purposes of this specification, a general reference to memory refers to memory accessible to the processor, including internal memory or removable memory plugged into the device and memory within the processor itself. For example, the memory can be any non-transitory computer readable medium on which computer readable instructions (e.g., computer program instructions) are stored that can be executed by the processor.

[0041] The term "electronic device" as used hereinafter refers to any one or all of a handheld device, a mobile phone, a wearable device, a personal data assistant (PDA), a tablet, a smartbook, a palmtop computer, a barcode reader, a scanner, a marker reader, an imager, a radio frequency identification (RFID reader or interrogator), an on-board computer, a wearable barcode scanner, a wearable marker reader, a point-of-sale (POS) terminal, a headset device, a programmable logic controller (PLC), a programmable automation controller (PAC), an industrial computer, a laptop computer, a desktop computer, a personal computer, and similar electronic devices equipped with at least one processor configured to perform various operations described herein.

[0042] For the sake of brevity, the terms "computing platform" or "host device" or "server" or "supervisor device" are used interchangeably herein to describe various embodiments. The term "server" can be used herein to refer to any computing device or distributed network of computing devices capable of functioning as a server, such as a host exchange server, web server, mail server, document server, or any other type of server. A server can be a dedicated computing device or a computing device that includes a server module (e.g., an application that runs that can cause the computing device to operate as a server). The server module (e.g., server application) can be a full-featured server module or a light or secondary server module (e.g., light or secondary server application) structured to provide synchronization services in a dynamic database on a computing device. The light or secondary server can be a scaled down version of a server-type functionality that can be implemented on a computing device such as a smartphone, thereby enabling it to function as an internet server (e.g., enterprise email server) only when needed to provide the functionality described herein.

[0043] In some example embodiments, the server can correspond to any of an industrial computer, a cloud computing based platform, an external computer, a standalone computing device, etc. In some example embodiments, the host device or computing platform can also refer to any electronic device as described herein. In some example embodiments, the server can include an access point or gateway device that can be capable of communicating directly with one or more electronic devices and can also be capable of establishing service (e.g., an internet service provider) communication with a network (either directly or alternatively indirectly via a communication network such as the internet). In some example embodiments, the server can manage the deployment of one or more electronic devices throughout a physical environment. In some example embodiments, the server can refer to a network establishing service that includes a distributed system, where a plurality of operations are performed by utilizing a plurality of computing resources deployed on a network and / or cloud based platform or cloud based service such as any of a software based service (SaaS), infrastructure based service (IaaS), or platform based service (PaaS), etc.

[0044] In some example embodiments, the term "server" can be used herein to refer to a programmable logic controller (PLC), programmable automation controller (PCC), industrial computer, desktop computer, personal data assistant (PDA), laptop computer, tablet, smartbook, palmtop computer, personal computer, smartphone, headset, smartwatch, and similar electronic devices equipped with a processor configured at least to perform the various operations described herein. Devices such as smartphones, tablets, headsets, and smartwatches are often collectively referred to as mobile devices.

[0045] The components shown in the figures represent components that can or can not be present in various embodiments of the application described herein, such that embodiments can include fewer or more components than those shown in the figures without departing from the scope of the application.

[0046] Various example embodiments described herein relate to techniques for improving performance of workflow operation execution. According to some examples, a workflow operation can be performed based on a voice dialogue that can include an exchange of voice prompts and voice responses between an operator and a voice control device used by the operator. An example workflow operation can be an operation for performing picking of various items in a materials handling environment (e.g., warehouse, inventory, etc.). According to some example embodiments, a machine learning model can be used to identify a performance state associated with execution of one or more steps of a workflow operation. The performance state is indicative of a progress and / or performance of the workflow operation execution compared to an expected performance metric (e.g., efficiency, throughput, turn-around time, etc.). For example, in one example, a machine learning model can be used to predict whether a picking operation performed by an operator is a slow picking operation. According to various example embodiments described herein, a prediction of the performance state can be performed at an early stage of execution of a picking operation step, i.e., prior to completion of the workflow operation. In some examples, early identification of the performance state enables an operator to perform a corrective action “on the fly” (i.e., while completing the workflow operation). Further, according to some example embodiments, based on the identified performance state, contextual suggestions can be provided to an operator performing a workflow operation. The contextual suggestions can be indicative of activities that can be performed to improve the performance state of the workflow operation execution. In some examples, performance states of operations performed by various operators can be displayed as predicted metrics on a dashboard in real-time and used by supervisors for improving productivity of operators in a work environment. For example, for picking operations, the predicted metrics can include a predicted picking rate, a context / reason for slow picking for each pick, daily / weekly reports, etc. According to some examples, contextual suggestions can be provided to improve overall execution of a workflow operation according to a workflow state associated with a step of the workflow operation. According to various example embodiments described herein with reference to FIGS. 1-8, additional details are described related to identifying performance states and improving execution of workflow operations. Figures 1 to 12 According to various example embodiments described herein, additional details are described related to identifying performance states and improving execution of workflow operations.

[0047] Figure 1 A workflow execution system 100 is shown that includes an example network architecture that can include one or more devices and subsystems that can be configured to implement some of the embodiments discussed herein. For example, the workflow execution system 100 can include a server 160 that can include, for example, a processor 162, a memory 164, and a network interface 166. The server 160 can be configured to implement some of the embodiments discussed herein. For example, the server 160 can be configured to implement a machine learning model 170 that can be used to identify a performance state associated with execution of one or more steps of a workflow operation. The performance state is indicative of a progress and / or performance of the workflow operation execution compared to an expected performance metric (e.g., efficiency, throughput, turn-around time, etc.). For example, in one example, the machine learning model 170 can be used to predict whether a picking operation performed by an operator is a slow picking operation. According to various example embodiments described herein, a prediction of the performance state can be performed at an early stage of execution of a picking operation step, i.e., prior to completion of the workflow operation. In some examples, early identification of the performance state enables an operator to perform a corrective action “on the fly” (i.e., while completing the workflow operation). Further, according to some example embodiments, based on the identified performance state, contextual suggestions can be provided to an operator performing a workflow operation. The contextual suggestions can be indicative of activities that can be performed to improve the performance state of the workflow operation execution. In some examples, performance states of operations performed by various operators can be displayed as predicted metrics on a dashboard in real-time and used by supervisors for improving productivity of operators in a work environment. For example, for picking operations, the predicted metrics can include a predicted picking rate, a context / reason for slow picking for each pick, daily / weekly reports, etc. According to some examples, contextual suggestions can be provided to improve overall execution of a workflow operation according to a workflow state associated with a step of the workflow operation. According to various example embodiments described herein with reference to FIGS. 1-8, additional details are described related to identifying performance states and improving execution of workflow operations. Figures 2 to 4The circuit, server, or database, etc. disclosed in the middle (not shown). The server 160 can include any suitable web server and / or other type of processing device. In some embodiments, the server 160 can receive requests and transmit information or indications regarding such requests to the operator devices 110A-110N and / or one or more supervisor devices 150. The operator devices 110A-110N referred to herein can correspond to electronic devices that operators (e.g., workers) in a work environment can use when performing various tasks. Further, the supervisor devices 150 referred to herein can correspond to electronic devices used by supervisors of the operators in a work environment. In one example, the work environment can correspond to a warehouse or inventory and the supervisor can be a warehouse manager.

[0048] In some example embodiments, the server 160 can communicate with one or more operator devices 110A-110N and / or one or more supervisor devices 150 via a network 120. In this regard, the network 120 can include any wired or wireless communication network, including, for example, wired or wireless local area networks (LANs), personal area networks (PANs), metropolitan area networks (MANs), wide area networks (WANs), etc., as well as any hardware, software, and / or firmware required to implement the same. For example, the network 120 can include cellular telephone, 802.11, 802.16, 802.20, and / or WiMax networks. In some embodiments, Bluetooth can be used to communicate between devices. Further, the network 120 can include public networks such as the Internet, private networks such as an intranet, or a combination of them, and can utilize a variety of networking protocols now available or later developed including, but not limited to, TCP / IP based networking protocols.

[0049] In some example embodiments, the network 103 can include, but is not limited to, a wireless fidelity (Wi-Fi) network, a picocell network, a personal area network (PAN), Zigbee, and a Scatternet. In some examples, the network 103 can correspond to a short-range wireless network through which the operator devices 102-10N can communicate with one another using one or more communication protocols such as, but not limited to, Wi-Fi, Bluetooth, Bluetooth Low Energy (BLE), Zigbee, ultrasound frequency-based networks, and Z-Wave. In some examples, the network 103 can correspond to a network in which the plurality of electronic devices 102-10N can communicate with one another using other various wired and wireless communication protocols such as Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), and 2G, 3G, or 4G communication protocols. In some examples, the network 103 can correspond to any communication network such as, but not limited to, LORA, cellular (NB IoT, LTE-M, leaky feeder coaxial cable, etc.).

[0050] In some exemplary embodiments, operator devices 110A-110N, supervisor devices 150, and / or server 160 may each be implemented as computing devices, such as personal computers and / or other networked devices, such as cellular phones, tablet computers, mobile devices, point-of-sale terminals, inventory management terminals, etc. Figure 1 The description of "N" components is for illustrative purposes only. Furthermore, although in Figure 1 Only one supervisor device 150 is shown, but in some embodiments, many or more supervisor devices 150 may be connected to the system. Furthermore, the workflow execution system 100 may include any number of users, operator devices, and / or supervisor devices. In one embodiment, operator devices 110A-110N and / or supervisor devices 150 may be configured to display an interface on the respective device's monitor for viewing, creating, editing, and / or otherwise interacting with the server. According to some embodiments, server 160 may be configured to display an interface on server 160's monitor for viewing, creating, editing, and / or otherwise interacting with information on server 160. In some embodiments, the interface of operator devices 110A-110N and / or supervisor devices 150 may differ from the interface of server 160. Various components of the system may execute on one or more of operator devices 110A-110N, supervisor devices 150, or server 160. The workflow execution system 100 may also include additional client devices and / or servers, etc.

[0051] According to some exemplary embodiments, operator devices 110A-110N may include, for example, but not limited to, electronic devices 102 (e.g., mobile devices, PDAs, etc.) and voice control devices 104 (e.g., headsets, wearable headsets, etc.). In this respect, an operator in the work environment can use electronic devices 102 and / or voice control devices 104 to perform one or more operations in the work environment. For example, in some exemplary embodiments, operator devices 110A-110N may be used by an operator to perform a workflow operation that may include one or more tasks. In this respect, in some examples, a workflow operation may include a sequence or series of steps to be performed by an operator. In some exemplary embodiments, one or more steps of a workflow operation may be provided to the operator on operator devices 110A-110A in the form of voice-guided instructions or instructions based on a graphical user interface (GUI).

[0052] For example, in a work environment (e.g., warehouse, industrial environment, distribution center, etc.), operators can use electronic devices 102 that can be preconfigured with an application (e.g., mobile application) to perform workflow operations. For example, in some examples, operators can use these devices (i.e., operator devices 110A-110N, electronic devices) for automatic identification of information and data capture and to improve productivity in the work environment. In some examples, the application can be used to perform various steps of a workflow operation. According to some example embodiments, the application can be installed on at least one of the electronic devices 102 and the voice controlled device 104 and can be used to generate instructions for the operator at each step of the workflow operation. These instructions can be provided on the electronic devices 102 and / or the voice controlled device 104.

[0053] According to some example embodiments, the voice controlled device 104 can be used to provide instructions to the operator in the form of "voice prompts" to perform various activities in the work environment. For example, in one example, for a pick workflow operation, the operator can be provided with instructions in the form of voice prompts on the voice controlled device 104 for picking various items in inventory. In this case, the voice prompts can include instructions for the operator such as, but not limited to, "arrive at the location of inventory," "confirm the check digit associated with the location," "identify the item from a number of items," "confirm the stock keeping unit (SKU) associated with the item," "pick the item," "move to the next location," and the like. Further, in some example embodiments, the electronic devices 102 can be configured to provide instructions to the operator in the form of visuals, i.e., instructions that can be displayed on the GUI of the electronic devices 102. Thus, the operator can perform the steps of the workflow operation based on the instructions provided in the voice prompts and / or visual prompts. Further, the electronic devices 102 and / or the voice controlled device 104 can be configured to receive responses from the operator to the instructions. For example, as the operator performs the tasks, the operator can provide "voice responses" and / or GUI input based responses on the voice controlled device 104 and / or the electronic devices 102, respectively.

[0054] Exemplarily, the operator devices 110A-110N are communicatively coupled through the network 103. Similarly, according to some example embodiments, the electronic devices 102 can be communicatively coupled to the voice controlled device 104 via the network 103. For example, the voice controlled device 104 can be communicatively coupled to the electronic devices 102 through a Bluetooth communication based network. In this regard, the electronic devices 102 can use the Bluetooth network to exchange data and various commands with the voice controlled device 104.

[0055] In some examples, the voice-based instructions and the vision-based instructions of a workflow task can be concurrently provided on the voice-controlled device 104 and the electronic device 102, respectively. In this regard, the execution state of the workflow on the electronic device 102 and / or the voice-controlled device 104 can be synchronized such that the operator can provide either a voice response and / or a GUI-based input in response to the voice prompts and / or vision instructions for a common step of a workflow operation to cause the workflow operation to move to the next state on both the voice-controlled device 104 and the electronic device 102.

[0056] According to some example embodiments, the operator devices 110A-110N can receive a file including one or more workflows to be executed on the operator devices 110A-110N. In this regard, according to some example embodiments, a workflow operation can be executed on the operator devices 110A-110N (e.g., the electronic device 102 and / or the voice-controlled device 104) based on message exchanges between the devices. In some example embodiments, the operator devices 110A-110N can receive a file including one or more workflows from the server 160.

[0057] According to some example embodiments, the electronic device 102, the voice-controlled device 104, the operator devices 110A-110N, the supervisor device 150, and / or the server 160 can include one or more input devices, including but not limited to a keyboard, a number pad, a mouse, a touch-sensitive display, a navigation key, a function key, a microphone, a voice recognition component, any other mechanism capable of receiving an input from a user, or any combination thereof. Furthermore, the operator devices 110A-110N, the supervisor device 150, and / or the server 160 can include one or more output devices, including but not limited to a display, a speaker, a tactile feedback mechanism, a printer, a light, any other mechanism capable of presenting output to a user, or any combination thereof.

[0058] The operator devices 110A-110N, the supervisor device 150, and / or the server 160 can include components for monitoring and / or collecting information about a user or an external environment in which a component is placed. For example, the operator devices 110A-110N, the supervisor device 150, and / or the server 160 can include sensors, scanners, and / or other monitoring components. In some embodiments, scanners can be used to determine the presence of certain individuals or items. For example, in some embodiments, the components can include scanners, such as optical scanners, RFID scanners, and / or other scanners configured to read human and / or machine-readable indicia physically associated with items.

[0059] Figure 2A schematic block diagram of circuitry 200 is shown, some or all of which can be included in, for example, electronic device 102, voice-controlled device 104, operator devices 110A-110N, supervisor device 150, and / or server 160. Any of the above-described systems or devices can include circuitry 200 and can be configured to perform the functions of circuitry 200 described herein independently or in conjunction with other devices in network 120. As shown, according to some example embodiments, circuitry 200 can include various means, such as a processor 210, a memory 220, a communication module 240, and / or an input / output module 250. In some embodiments, a workflow database 300 and / or a workflow system 400 can also or alternatively be included. As referred to herein, a “module” includes hardware, software, and / or firmware configured to perform one or more particular functions. In this regard, means of circuitry 200 as described herein can be embodied as, for example, circuitry that includes computer-readable program instructions stored on a non-transitory computer-readable medium (e.g., memory 220) and executable by a properly configured processing device (e.g., a processor 210), a hardware element (e.g., a properly programmed processor, a combinational logic circuit, and / or the like), a computer program product, or some combination thereof. Figure 2 As shown, according to some example embodiments, circuitry 200 can include various means, such as a processor 210, a memory 220, a communication module 240, and / or an input / output module 250. In some embodiments, a workflow database 300 and / or a workflow system 400 can also or alternatively be included. As referred to herein, a “module” includes hardware, software, and / or firmware configured to perform one or more particular functions. In this regard, means of circuitry 200 as described herein can be embodied as, for example, circuitry that includes computer-readable program instructions stored on a non-transitory computer-readable medium (e.g., memory 220) and executable by a properly configured processing device (e.g., a processor 210), a hardware element (e.g., a properly programmed processor, a combinational logic circuit, and / or the like), a computer program product, or some combination thereof.

[0060] Processor 210 may, for example, be embodied as various means including one or more microprocessors, one or more instances of one or more multi-core processors, one or more controllers, one or more instances of one or more digital signal processors, one or more instances of one or more co-processors, one or more controllers, one or more computers, various other processing elements, including integrated circuits such as, for example, an ASIC (application specific integrated circuit) or FPGA (field programmable gate array), or some combination thereof. Accordingly, although illustrated in Figure 2 the singular in FIG. 1, in some embodiments, processor 210 includes a plurality of processors. The plurality of processors can be embodied on a single computing device or can be distributed among a plurality of computing devices collectively configured to function as circuitry 200. The plurality of processors are in operative communication with one another and are collectively configured to perform one or more functions of circuitry 200 as described herein. In an example embodiment, processor 210 is configured to execute instructions stored in memory 220 or otherwise accessible to processor 210. These instructions may, when executed by processor 210, cause circuitry 200 to perform one or more of the functions of circuitry 200 as described herein.

[0061] Whether processor 210 is configured by a hardware approach, a firmware / software approach, or a combination thereof, the processor can include an entity capable of performing operations according to embodiments of the present application while configured accordingly. Thus, for example, when processor 210 is embodied as an ASIC, FPGA or the like, processor 210 can include specifically configured hardware for conducting one or more operations described herein. Alternatively, as another example, when processor 210 is embodied as an executor of instructions (such as can be stored in memory 220), the instructions can specifically configure processor 210 to perform one or more algorithms and operations described herein, such as in conjunction with a Figures 1 to 12 discussion of those algorithms and operations.

[0062] Memory 220 can include, for example, volatile memory, non-volatile memory, or some combination thereof. Although illustrated in Figure 2 the present disclosure as a single memory, memory 220 can comprise multiple memory components. The multiple memory components can be implemented on a single computing device or distributed across multiple computing devices. In various embodiments, memory 220 can include, for example, a hard disk, random access memory, cache memory, read only memory (ROM), erasable programmable read only memory (EPROM) and electrically erasable programmable read only memory (EEPROM), flash memory, a magnetic tape, a magnetic disk drive or other magnetic storage device, an optical disk drive, a compact disk read only memory (CD-ROM), a digital versatile disk read only memory (DVD-ROM), a cassette, a floppy disk, a punch card, a paper tape, a ROM, or some combination thereof. Memory 220 can be configured to store information, data (including item data and / or profile data), applications, instructions, etc. for use by circuit 200 in executing various functions in accordance with exemplary embodiments of the present application. For example, in at least some embodiments, memory 220 is configured to buffer input data for processing by processor 210. Additionally or alternatively, in at least some embodiments, memory 220 is configured to store program instructions for execution by processor 210. Memory 220 can store information in the form of static and / or dynamic information. This stored information can be stored and / or used by circuit 200 in the course of executing its functions.

[0063] The communication module 240 can be embodied as any device or apparatus that is embodied in circuitry, hardware, a computer program product, or a combination thereof that includes computer readable program instructions stored on a computer readable medium (e.g., the memory 220) and executed by a processing device (e.g., the processor 210) and configured to receive and / or transmit data from / to another device and / or network, such as the second circuit 200. In some embodiments, the communication module 240 (as with other components discussed herein) can be embodied at least partially as or otherwise under the control of the processor 210. In this regard, the communication module 240 can be in communication with the processor 210, such as via a bus. The communication module 240 can include, for example, an antenna, a transmitter, a receiver, a transceiver, a network interface card, and / or supporting hardware and / or firmware / software for enabling communications with another computing device. The communication module 240 can be configured to receive and / or transmit any data that can be stored by the memory 220 using any protocol that can be used for communications between computing devices. The communication module 240 can additionally or alternatively be in communication with the memory 220, the input / output module 250, and / or any other component of the circuit 200, such as via a bus.

[0064] The input / output module 250 can be in communication with the processor 210 to receive indications of user input and / or to provide audible, visual, mechanical, or other output to a user (e.g., an employee and / or a customer). In connection with Figures 1 to 12 Some example visual outputs that can be provided to a user by the circuit 200 are discussed. Thus, the input / output module 250 can include, for example, support for a keyboard, a mouse, a joystick, a display, a touchscreen display, a microphone, a speaker, an RFID reader, a barcode reader, a biometric scanner, and / or other input / output mechanisms. As compared to embodiments in which the circuit 200 is implemented as an end user machine (e.g., a remote worker device and / or an employee device) or other type of device designed for sophisticated user interaction, aspects of the input / output module 250 can be reduced in embodiments in which the circuit 200 is implemented as a server or database. In some embodiments (as with other components discussed herein), the input / output module 250 can even be eliminated from the circuit 200. Alternatively, at least some aspects of the input / output module 250 can be embodied on a device used by a user that is in communication with the circuit 200, such as in embodiments in which the circuit 200 is embodied as a server or database. The input / output module 250 can be in communication with the memory 220, the communication module 240, and / or any other component, such as via a bus. One or more input / output modules and / or another component can be included in the circuit 200.

[0065] The workflow database 300 and the workflow system 400 can also or alternatively be included or configured to perform the functions discussed herein related to workflows and / or identify performance states associated with the performance of workflows. In some embodiments, some or all of the functions of generating workflows and / or information for workflows and / or performance states associated with the performance of workflows can be performed by the processor 210. In this regard, the example processes and algorithms discussed herein can be performed by the at least one processor 210, the workflow database 300, and / or the workflow system 400. For example, a non-transitory computer-readable medium can be configured to store firmware, one or more applications, and / or other software including instructions and other computer-readable program code portions, executable by each processor of the circuit 200 (e.g., the processor 210, the workflow database, and / or the workflow system) to control the components of the circuit 200, thereby implementing various operations, including the examples shown above. Thus, a series of computer-readable program code portions are embodied in one or more computer program products and are usable with a computing device, server, and / or other programmable devices to produce a machine implemented process.

[0066] As shown, according to some example embodiments, a workflow database 300 can be provided that includes various relevant information for the workflow execution system. For example, as shown, in this embodiment, the workflow database 300 can include employee profile data 311, task data 312, historical voice conversation data 313, performance state data 314, and contextual suggestion data 315. Various other data can be included in the workflow database 300. As additional tasks are performed, additional information (e.g., performance information) about the tasks and employees can be received by the circuit 200, which can be stored in the workflow database 300. Further, additional information related to various products, services, workflow operations (related to work environments) can be stored in the workflow database 300 for use. Additionally or alternatively, the workflow database 300 can include contextual suggestion data 315 that provides any additional information needed by the workflow system 400 in analyzing inputs and requests and generating appropriate responses. Figure 3 Figure 3

[0067] ​​For example, workflow system 400 can be configured to analyze multiple data sets (e.g., various combinations including employee profile data, task data, historical voice dialog data, performance state data, contextual suggestion data, etc.), such as the data in workflow database 300. As such, workflow system 400 can support multiple algorithms, including those discussed below with respect to employee profile data, task data, historical voice dialog data, performance state data, contextual suggestion data, etc., such that a selected algorithm can be selected at runtime. Moreover, the present configuration can enable flexibility to configure additional contextual aspects.

[0068] Figure 4 A schematic block diagram of a workflow system 400 for improving workflow execution is shown in accordance with example embodiments. In some embodiments, referring to Figure 4 , workflow system 400 can include a contextual recognition module 420, a machine learning engine 430, and a communication interface 440, all of which can be in communication with workflow database 300. Workflow system 400 can receive one or more inputs or requests (e.g., voice commands) and can generate an appropriate response. For example, workflow system 400 can generate a voice prompt including instructions for performing steps of a workflow operation in the form of voice commands. Workflow system 400 can use any of the algorithms or processes disclosed herein to receive requests / inputs and generate responses. In some other embodiments, such as when circuit 200 is embodied in server 160, supervisor device 150, and / or operator devices 110A-110N, workflow system 400 can be located in another circuit 200 or another device, such as another server 160, supervisor device 150, and / or operator devices 110A-110N. Workflow system 400 can be configured to access data corresponding to one or more employees, performance states, execution of tasks of one or more workflow operations, warehouse data, etc., and generate one or more responses and / or indications.

[0069] Referring to Figure 4 , whether used locally or over a network, workflow system 400 can be used to analyze workflow execution, identify contextual information based on operator voice dialogs, create suggestions / notifications associated with tasks and subtasks, and notify supervisors or managers of relevant tasks / subtasks and performance states associated with operator execution of a workflow. The system can receive multiple inputs 410, 415 from circuit 200 and process the inputs within workflow system 400 to produce an output 450. In accordance with various example embodiments described herein, output 450 can indicate a performance state associated with execution of a workflow task. In other words, output 450 can indicate performance of execution of a workflow task.

[0070] As described in accordance with various example embodiments, each workflow activity can be performed based on a voice dialog (i.e., an exchange of voice prompts and voice responses) between the operator and the voice control device 104. In this regard, each step of the workflow can be predefined in accordance with the described example embodiments. In other words, the workflow can include a set of predefined steps / tasks to be performed to complete the workflow. As previously described, the instructions for performing each of these steps can be provided as voice prompts. Thus, there can be a fixed type / number of voice prompts in accordance with the type of workflow (e.g., but not limited to, a sequential pick workflow, a cycle count workflow, a stock replenishment workflow, etc.), and the voice response expected for each of the voice prompts can also be predefined. To this end, each voice dialog associated with a workflow can include one or more contextual key words (or set of words, phrases, etc.) that can be predefined for that workflow. Further, in some examples, the contextual words can be defined in accordance with a context associated with that step of the workflow. For example, an example of a contextual key word can be "location," which can be provided as a voice prompt to the operator to find the operator's current location. Similarly, another example of a contextual key word can be "ready," which can be provided as a voice response by the operator to indicate an affirmative in response to a voice prompt.

[0071] Further, in accordance with various example embodiments described herein, the location of the contextual key words in the voice dialog, the timestamp of the occurrence of the contextual key words, the frequency of occurrence of the contextual key words can be predefined for the workflow. For example, as an example, for a workflow, it can be known that the contextual word "line" should generally occur at the beginning of the voice dialog during the execution of the workflow. Similarly, as another example, the contextual key word "ready" can be provided by the operator at a predefined timestamp, or after a predefined time interval (e.g., within 10 seconds) after a particular voice prompt. In accordance with another example, the contextual key word can include a set of words (e.g., three alpha-numeric digits) that are provided one after the other in a predefined sequence in the voice response. In another example, the contextual key word can also include a set of words that occur in a defined pattern in the voice dialog.

[0072] According to various example embodiments described herein, the context recognition module 420 can receive data files comprising one or more voice conversations as input (410, 415...41n). The context recognition module 420 can parse the input (410, 415...41n) to identify a set of contextual keywords and / or contextual parameters associated with one or more of the contextual keywords (e.g., location of the contextual keyword in the voice conversation, timestamp of the occurrence of the contextual keyword, frequency of occurrence of the contextual keyword, etc.). In this regard, in example embodiments, the input (410, 415...41n) can be received as unstructured data from the workflow database 300 in a log file of the voice conversation. In some example embodiments, the context recognition module 420 can identify the set of contextual keywords and / or associated contextual parameters by performing a frequency analysis while parsing the input voice conversation. Further, the contextual keywords and / or contextual parameters (e.g., logical states associated with the contextual keyword, frequency of occurrence, frequency of repetition, timestamp, and / or time period of occurrence of the contextual keyword) can be stored in the workflow database 300 for training the machine learning model.

[0073] Further, according to some example embodiments, when the workflow system 400 receives the input 410, 415, the context recognition module 420 can determine other additional information indicative of a context associated with the workflow. For example, in some examples, the context recognition module 420 can determine information such as operator profile data (e.g., which employee is associated with the input 410, 415), operator historical performance data (e.g., how the employee has handled tasks associated with the input 410, 415 in the past), task data (e.g., which task is associated with the input 410, 415), preference data of the system, and which request or indication was received as the input 410, 415, etc. These inputs can give context to the machine learning engine 430 of the workflow system to determine an output indicative of a performance state associated with the performance of the workflow operation task.

[0074] According to some example embodiments, one or more patterns in the contextual key terms and / or contextual parameters can be identified to train a machine learning model. In this regard, in some example embodiments, the machine learning engine 430 can convert unstructured data of a data file into a structured data format representing a matrix that includes each of the contextual key terms from a voice conversation in a row and its associated contextual parameters in a column. According to various example embodiments described herein, the machine learning engine 430 can receive the structured data (i.e., a set of contextual key terms and / or associated contextual parameters) as input features. The machine learning engine 430 can generate a machine learning model that can be trained using the input features. In this regard, the machine learning engine 430 using the machine learning model can output a classification indicative of a performance state associated with the performance of a workflow task.

[0075] According to some example embodiments, the machine learning engine 430 can employ a support vector machine (SVM) classifier to determine one or more classifications, one or more correlations, one or more expressions, one or more inferences, one or more patterns, one or more features, and / or other learned information related to the input features (e.g., structured data output by the context identification module 420). In another example embodiment, the machine learning engine 430 can employ one or more machine learning classification techniques associated with a Bayesian machine learning network, a binary classification model, a multi-class classification model, a linear classifier model, a quadratic classifier model, a neural network model, a probabilistic classification model, a decision tree, and / or one or more other classification models. The machine learning model (e.g., classification model, machine learning classifier, etc.) employed by the machine learning engine 430 can be explicitly trained (e.g., via training data) and / or implicitly trained (e.g., via extrinsic data received by the machine learning model). For example, the machine learning model (e.g., classification model, machine learning classifier, etc.) employed by the machine learning engine 430 can be trained by training data (i.e., input features) that includes a set of contextual key terms and / or contextual parameters associated with the contextual key terms.

[0076] According to various example embodiments described herein, the machine learning engine 402 can generate a machine learning model that can perform an analysis (e.g., a regression or decision tree analysis) using (a) input features (i.e., a set of contextual keyword phrases and / or contextual parameters identified from one or more historical voice conversations), and (b) an initial few instances of voice prompts and voice responses of the ongoing task of the workflow in execution for providing an output (referred to herein for brevity as first voice prompts and first voice responses). According to some example embodiments, to generate the output, the machine learning engine 430 can identify a pattern associated with one or more of (a) the contextual keyword phrases, (b) the contextual parameters associated with the contextual keyword phrases, (c) the initial instances of the voice prompts of the ongoing task of the workflow, and (d) the initial instances of the voice responses of the ongoing task of the workflow.

[0077] For example, a historical voice conversation associated with a workflow task that can be provided as input (410, 415...41n) to the workflow system 400 can include a plurality of voice prompts including instructions for reaching a location in a warehouse and picking up an item from the location. For example, in one example, the voice prompts can be "Aisle, HU color is purple, check digit middle," "bravo golf One Six alpha Zero Two," etc. In addition, the voice conversation can also include voice responses that can be provided by the operator in response to the voice prompts. In some examples, the voice responses can indicate any of a confirmation of performing the step of the workflow task, a data value, etc. For example, in one example, the voice responses provided by the operator can include "Aisle," "Ready," "8," "4," etc. In accordance with the various example embodiments described herein, the machine learning engine 430 can identify one or more patterns associated with context keywords identified from the voice conversation by analyzing the voice conversation. The identification of the one or more patterns can be based on a context associated with each step of the workflow. For example, the machine learning engine 430 can identify the word "Aisle" that is typically present at the beginning of the voice prompts. In another example, the machine learning engine 430 can identify the voice response "Ready" that is typically provided by the operator within 30 seconds of receiving a previous voice prompt. Another example pattern can be that the word "Bravo" typically occurs three times during the performance of the workflow task. Accordingly, the machine learning engine 430 can identify such patterns and use these patterns associated with the context keywords to train a machine learning model that can be used to generate an output. The output of the machine learning engine 430 can be related to a performance state associated with the performance of the ongoing task of the workflow. In this regard, the output of the machine learning model employed by the machine learning engine 430 can indicate a progress of the task performance (e.g., normal operation, delayed operation, incorrect operation). In some examples, the output of the machine learning engine 430 can be a classification that indicates the progress of the workflow task performance. For example, for an item pick-up operation, the output of the machine learning engine 430 can indicate a classification such as normal pick-up, delayed pick-up, incorrect pick-up, etc.

[0078] In addition, as previously described, the machine learning engine 430 can identify the performance state during the course of the performance of the workflow task and prior to completion. In addition, in response to the identification of the performance state, the machine learning engine 430 can also output a suggestion that can be implemented by the operator to improve the performance state associated with the performance of the workflow task. In some examples, the suggestion can be provided to the operator depending on the step or task that the operator is facing issues moving forward in performing the workflow.

[0079] Figure 5An exemplary voice control device 500 is shown in accordance with one exemplary embodiment. In Figure 5 In the illustrated embodiment, the voice control device 500 can correspond to a headset that can include a wireless-enabled voice recognition device that utilizes a hands-free profile.

[0080] In accordance with some exemplary embodiments, the headset can be substantially similar to the headsets disclosed in U.S. Provisional Patent Application No. 62 / 097,480, filed December 29, 2014, U.S. Provisional Patent Application No. 62 / 101,568, filed January 9, 2015, and U.S. Patent Application No. 14 / 918,969, and the disclosures therein are hereby incorporated by reference in their entirety.

[0081] In accordance with exemplary embodiments, as shown, the voice control device 500 can include an electronics module 502. In this embodiment, some elements can be incorporated into the electronics module 502 rather than the headset 503 to provide for long battery life consistent with long work shifts. For example, one or more components of the circuit 200 can be incorporated in the electronics module 502 and / or the headset 503. In some exemplary embodiments, the electronics module 502 can be remotely coupled to a lightweight and comfortable headset 503 that is secured to the worker's head via a headband 504. In some exemplary embodiments, the headband 504 can be a band designed to fit over the worker's head, in the ear, over the ear, or otherwise designed to support the headset. The headset 503 can include one or more speakers 505 and can also include one or more microphones. For example, in Figure 5 In the illustrated embodiment, the headset 503 includes microphones 506, 507. In accordance with some exemplary embodiments, the microphone 507 can provide noise cancellation by continuously listening and blocking ambient sounds to enhance voice recognition and optionally provide noise cancellation. In some embodiments (not shown), the electronics module 502 can be integrated into the headset 503 rather than remotely coupled to the headset 503. Various configurations of the voice control device 500 can be used without departing from the intent of the present disclosure.

[0082] In some exemplary embodiments, the electronics module 502 can be used to offload several components of the headset 503 to reduce the weight of the headset 503. In some embodiments, a rechargeable or long-life battery, a display, a keypad, One or more of an antenna and printed circuit board assembly (PCBA) electronics can be included in the electronics module 502 and / or otherwise incorporated into the voice control device 500.

[0083] In Figure 5In the illustrated embodiment, the headset 503 can be coupled to the electronics module 502 via a communication link such as a small audio cable 508, but can alternatively communicate with the electronics module 502 via a wireless link. In example embodiments, the headset 503 can be small in profile. For example, in some embodiments, the headset 503 can be minimalist in appearance, such as a Bluetooth earpiece / headset.

[0084] According to some example embodiments, the electronics module 502 can be configured to be used with various types of headsets 503. In some example embodiments, the electronics module 502 can read a unique identifier (I.D.) of the headset 503, which can be stored in the circuitry (e.g., circuitry 200) of the voice controlled device 500 and can also be used to electronically couple the speaker and microphone to the electronics module 502. In one embodiment, the audio cable 508 can include a plurality of conductors or communication lines for signals, which can include speaker+, speaker-, ground digital, microphone, auxiliary microphone, and microphone ground. In some examples, the electronics module 502 can utilize a user configurable attachment 509, such as a plastic ring, to attach to the user. For example, in Figure 5 In the illustrated embodiment, the electronics module 502 can be mounted to the worker's torso via a collar clip and / or lanyard. In some embodiments, the headset 503 can include a small lightweight battery, such as when a wireless link between the headset 503 and the electronics module 502 can be used, such as a Bluetooth type communication link. The communication link can provide wireless signals suitable for exchanging voice communications.

[0085] In some embodiments, a voice template for performing speaker dependent training of a speech recognition model can be stored locally in the electronics module 502 and / or headset 503 as part of the circuitry 200 to recognize voice interactions of the user and can convert the interactions into text based data and commands for interacting with applications running in the circuitry 200. For example, in one embodiment, the voice controlled device 500 can utilize a voice template to perform speech recognition. According to some example embodiments, the first few stages of speech recognition can be performed in the voice controlled device 500, with additional stages performed on the server 160. In further embodiments, raw audio can be transmitted from the voice controlled device 500 to the server 160, where the final stages of speech recognition can be completed. Alternatively, in some example embodiments, speech recognition can be performed on the voice controlled device 500.

[0086] Figure 6 An example block diagram of the electronics module 502 is shown in accordance with some embodiments of the present disclosure. Figure 6 The illustrated components can be in addition to Figure 2components of the circuit 200 shown can be part of the electronic module 502. In some embodiments, Figure 6 One or more of the components shown can be included in other portions of the electronic module 502 and / or the voice-controlled device (500, 104), the electronic device 102, the operator devices 110A-110N, the supervisor device 150, and / or the server 160.

[0087] In Figure 6 In the embodiment shown, the electronic module 502 can include a housing (such as a plastic housing) having a connector 510 that can mate with a complementary mating connector (not shown) on the audio cable 508. An internal path 511 can be used to communicate between the various components within the housing of the electronic module 502. In one embodiment, an input speech pre-processor (ISPP) 512 can convert input speech into pre-processed speech feature data. In some examples, an input speech encoder (ISENC) 513 can encode input speech for transmission to one or more other portions of the circuit 200 for reconstruction and playback and / or recording. Further, a raw input audio sample packet formatter 514 can transmit raw input audio to one or more other portions of the circuit 200 using an application layer protocol for facilitating communication between the voice terminal and the headset 503 as a transport mechanism. For the purposes of the transport mechanism, the formatter 514 can be abstracted as a codec type called input audio sample data (IASD). An output audio decoder (OADEC) 515 decodes encoded output speech and audio for playback in the headset 503. According to some example embodiments, a raw output audio sample packet reader 516 can operate to receive raw audio packets from one or more other portions of the circuit 200 using the transport mechanism. For the purposes of the transport mechanism, this formatter 514 can be abstracted as a codec type called output audio sample data (OASD). A command processor 517 can adjust headset hardware (e.g., input hardware gain levels) under the control of one or more other portions of the circuit 200. Further, in some example embodiments, a query processor 518 can allow one or more other portions of the circuit 200 to retrieve information about headset operating status and configuration. Further, the path 511 can also be coupled to a network circuit 519 to communicate with one or more other portions of the circuit 200 via wired or wireless protocols. In some examples, the ISPP 512, the ISENC 513, and the raw input audio formatter 514 can be sources of communication packets used in the transport mechanism; the OADEC 515 and the raw output audio reader 516 can be packet receivers. The command processor 517 and the query processor 518 are both packet receivers as well as sources (generally, they generate acknowledgement or response packets).

[0088] Figure 7 A schematic diagram 700 of an example electronic device (e.g., electronic device 102, operator device 110A-110N, supervisor device 150, etc.) is shown in accordance with example embodiments described herein. In some example embodiments, electronic device 102 can correspond to a mobile handheld device. Figure 7 A schematic block diagram of an example end user device, such as user equipment that can be an electronic device 102 used by an operator to perform one or more tasks of a workflow, is shown.

[0089] While Figure 7 A mobile handheld device is shown, but it should be understood that other devices can be Figure 1 described herein, and the mobile handheld device is merely shown to provide context for embodiments of the various embodiments described herein. To this end, the following discussion is intended to provide a brief, general description of an example of a suitable environment in which various embodiments can be implemented. While the description includes a general context of computer-executable instructions embodied on a machine-readable storage medium, those skilled in the art will recognize that the various embodiments also can be implemented in combination with other program modules and / or as a combination of hardware and software.

[0090] Generally, application programs (e.g., program modules) can include routines, programs, components, data structures, etc., that can perform particular tasks or implement particular abstract data types according to the example embodiments described herein. Moreover, those skilled in the art will appreciate that the methods described herein can be practiced with other system configurations, including single-processor or multiprocessor systems, minicomputers, mainframe computers, as well as personal computers, hand-held computing devices, microprocessor-based or programmable consumer electronics, and the like, each of which can be operatively coupled to one or more associated devices.

[0091] According to some example embodiments, the electronic device 102, the operator devices 110A-110N, and the voice-controlled device 104 can generally include various machine-readable media. Machine-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer-readable media can comprise computer storage media and communication media. Computer storage media can include volatile and / or non-volatile media, removable and / or non-removable media, implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media can include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer.

[0092] According to some example embodiments described herein, communication media generally embodies computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. In this context, the term “modulated data signal” can correspond to a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of the any of the above can also be included within the scope of computer readable media.

[0093] According to some example embodiments, the mobile handset can include a processor 702 for controlling and processing all onboard operations and functions. A memory 704 interfaces to the processor 702 for the storage of data and one or more application programs 706 (e.g., a video player software, a user feedback component software, etc.). Other applications can include voice recognition of predetermined voice commands that facilitate initiation of a user feedback signal. The application programs 706 can be stored in the memory 704 and / or firmware 708 and executed by the processor 702 from one or both of the memory 704 and / or firmware 708. The firmware 708 can also store startup code for execution during startup of the mobile handset. A communications component 710 interfaces to the processor 702 to facilitate wired / wireless communication with external systems, such as cellular networks, VoIP networks, etc. Here, the communications component 710 can also include suitable cellular transceiver 711 (e.g., a GSM transceiver) and / or unlicensed transceiver 713 (e.g., Wi-Fi, WiMAX) for corresponding signal communication. The mobile handset can be a device having mobile communication capabilities, such as a cellular telephone, a PDA, and a messaging-centric device. The communications component 710 also facilitates communication reception from land radio networks (e.g., broadcast), digital satellite radio networks, and Internet-based radio service networks.

[0094] The mobile handset can also include a display 712 (e.g., a display screen) for displaying text, images, videos, telephonic functions (e.g., caller ID functions), setup functions, and for user input. For example, the display 712 can also be referred to as a "screen" that can accommodate presentation of multimedia content (e.g., music metadata, messages, wallpaper, graphics, etc.). The display 712 can also display videos and can facilitate generation, editing, and sharing of video quotes. A serial I / O interface 714 is provided in communication with the processor 702 to facilitate wired and / or wireless serial communications (e.g., USB and / or IEEE 1384) through a hardwire connection and other serial input devices (e.g., a keyboard, keypad, and mouse). This supports, for example, updating and troubleshooting of the mobile handset. Audio functions are provided via an audio I / O component 716, which can include a speaker for outputting audio signals related to, for example, indicating that a user has pressed the correct key or combination of keys to initiate a user feedback signal. The audio I / O component 716 also facilitates input of audio signals through a microphone for recording data and / or telephonic voice data, and for inputting voice signals for a telephonic conversation.

[0095] The mobile handset can also include a slot interface 718 for accommodating a Subscriber Identity Module (SIM) or Universal SIM 720 form factor SIC (Subscriber Identity Component), and to interface the SIM card 720 with the processor 702. However, it is understood that the SIM card 720 can be manufactured into the mobile handset and can be updated by downloading data and software.

[0096] The mobile handset can also handle IP data traffic through the communication component 710 to accommodate IP traffic from an IP network, such as the Internet, an intranet, an extranet, a home network, a personal area network, etc., through a WAN or broadband cable provider. Thus, VoIP traffic can be utilized by the mobile handset, and IP-based multimedia content can be received in encoded form or decoded form.

[0097] A video processing component 722 (e.g., a camera) can be provided for decoding encoded multimedia content. The video processing component 822 can facilitate the generation, editing, and sharing of video offers. The mobile handset also includes a power source 724 in the form of a battery and / or AC power subsystem, which can interface to an external power system or charging equipment (not shown) through a power I / O component 726.

[0098] According to some example embodiments, the mobile handset can also include a video component 730 for processing received video content, as well as for recording and transmitting video content. For example, the video component 730 can facilitate the generation, editing, and sharing of video offers. In some example embodiments, a location tracking component 732 facilitates geographically locating the mobile handset. As described above, this can occur when the user initiates a feedback signal, either automatically or manually. According to some example embodiments, a user input component 734 facilitates the user initiating a quality feedback signal. In this regard, in some examples, the user input component 734 can also facilitate the generation, editing, and sharing of video offers. According to various example embodiments described herein, the user input component 734 can include such conventional input device technology as a keypad, keyboard, mouse, stylus, and / or touch screen.

[0099] Referring again to the application 706, a hysteresis component 736 can facilitate analyzing and processing hysteresis data for determining when to associate with an access point. A software trigger component 838 can be provided that facilitates triggering the hysteresis component 738 when a Wi-Fi transceiver 713 detects a beacon of an access point. A SIP client 740 enables the mobile handset to support the SIP protocol and to manage subscriber registration with a SIP registrar server. In some example embodiments, the application 706 can also include a client 742 that provides at least the ability to discover, play, and store multimedia content (e.g., music).

[0100] In some example embodiments, as described above, the mobile handset associated with the communication component 710 includes an indoor network radio transceiver 713 (e.g., a Wi-Fi transceiver). This functionality can support an indoor radio link for dual-mode GSM handsets, such as IEEE 802.11. In some example embodiments, the mobile handset can accommodate at least satellite radio service through a handset that can combine wireless voice and digital radio chipsets into a single handheld device.

[0101] Figure 8 A diagram illustrating another example of an electronic device 801 in accordance with another example embodiment described herein is shown. According to some example embodiments, Figure 8 The electronic device 801 shown can correspond to the electronic device 102, the operator devices 110A-110N, the supervisor device 150, and / or the server 160, as Figures 1 to 7 described with reference to

[0102] Reference is now made to Figure 8 which shows a block diagram for functions and operations performed in the example embodiments described. In some example embodiments, the electronic device 801 can provide networking and communication capabilities between wired or wireless communication networks and servers and / or communication devices. To provide additional context for various aspects thereof, Figure 8 and the following discussion is intended to provide a brief, general description of a suitable computing environment in which various aspects of the embodiments can be implemented to facilitate the establishment of transactions between entities and third parties. While the above description is in the general context of computer-executable instructions that can run on one or more computers, those skilled in the art will recognize that the various embodiments also can be implemented in combination with other program modules and / or as a combination of hardware and software.

[0103] According to the example embodiments described, program modules include routines, programs, components, data structures, etc., that perform particular tasks or implement particular abstract data types. Moreover, those skilled in the art will appreciate that the inventive methods can be practiced with other computer system configurations, including single-processor or multiprocessor computer systems, minicomputers, mainframe computers, as well as personal computers, hand-held computing devices, microprocessor-based or programmable consumer electronics, etc., each of which can be operatively coupled to one or more associated devices.

[0104] The illustrated aspects of the various embodiments can also be practiced in distributed computing environments where certain tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote memory storage devices.

[0105] In accordance with some example embodiments, computing devices generally include various media, which can include computer-readable storage media or communications media, which both terms are used herein differently from one another. Such media can further include any appropriate media then-existing or yet to be developed including storage media and communication media in any suitable form.

[0106] In accordance with some example embodiments, computer-readable storage media can be any available storage media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer-readable storage media can be implemented in connection with any method or technology for storage of information, such as computer-readable instructions, program modules, structured data, or unstructured data. Computer-readable storage media can include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or other tangible and / or non-transitory media which can be used to store the desired information. Computer-readable storage media can be accessed by one or more local or remote computing devices, e.g., via access requests, queries or other data retrieval protocols, for the information stored in the media.

[0107] In some examples, communication media can embody computer- readable instructions, data structures, program modules, or other structured or unstructured data in a modulated data signal, such as a carrier wave or other transport mechanism, and includes any information delivery or transport media. The term “modulated data signal” or signal refers to a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media.

[0108] With reference to Figure 8 Implementing various aspects described herein with respect to end-user devices can include an electronic device 801 that includes a processing unit 804, a system memory 806, and a system bus 808. The system bus 808 can be configured to couple the system components including, but not limited to, the system memory 806 to the processing unit 804. In some example embodiments, the processing unit 804 can be any of a variety of commercially available processors. To this end, in some examples, dual microprocessors and other multi-processor architectures can also be employed as the processing unit 804.

[0109] According to some example embodiments, system bus 808 can be any of several types of bus structures including a memory bus with or without a memory controller to which memory is attached, a peripheral bus to which various peripherals are attached, and a local bus that is used interconnect various elements of a computer. In some examples, system memory 806 can include read-only memory (ROM) 827 and random access memory (RAM) 812. According to some example embodiments, a basic input / output system (BIOS) containing the basic routines that help to transfer information between elements within the computing device 801, such as during start-up, is stored in nonvolatile memory 827, such as ROM, EPROM, EEPROM. RAM 812 can also include a high-speed RAM such as static RAM for caching data.

[0110] According to some example embodiments, computing device 801 can also include an internal hard disk drive (HDD) 814 (e.g., EIDE, SATA), which can also be configured for external use in a suitable chassis (not shown), a floppy disk drive (FDD) 816 (e.g., to read or write to a removable disk 818), and an optical disk drive 820 (e.g., to read a CD-ROM disk, or to read from or write to other high capacity optical media such as DVDs). In some examples, hard disk drive 814, disk drive 816, and optical disk drive 820 can be connected to system bus 808 by a hard disk drive interface 824, a disk drive interface 826, and an optical drive interface 828, respectively. According to some example embodiments, interface 824 for external drive implementations can include at least one or both of Universal Serial Bus (USB) and IEEE 1394 interface technology. Other external drive connection technology is also contemplated within the subject embodiments.

[0111] According to some example embodiments described herein, drives and their associated computer-readable media provide nonvolatile storage for data, data structures, computer-executable instructions, and the like. For electronic device 801, the drives and media accommodate any data in suitable digital formats. Although the above description of computer-readable media refers to a HDD, a removable disk, and a removable optical media such as a CD or DVD, those skilled in the art will appreciate that other types of media that can be read by electronic device 801, such as zip drives, magnetic cassettes, flash memory cards, tape, etc., can also be used in the example operating environment, and that yet other implementations can not employ any media at all. In addition, any of these media can contain computer-executable instructions for performing the methods disclosed herein.

[0112] In some example embodiments, a plurality of program modules can be stored in the drives and the RAM 812, including an operating system 830, one or more application programs 832, other program modules 834, and program data 836. In this regard, in some examples, all or portions of the operating system, applications, modules, and / or data can also be cached in the RAM 812. It is to be appreciated that various embodiments can be implemented with various commercially available operating systems or combinations of operating systems.

[0113] According to some example embodiments, a user can enter commands and information into the computing device 801 through one or more wired / wireless input devices, e.g., a keyboard, and a pointing device, such as a mouse 840. Other input devices (not shown) can include a microphone, an IR remote control, a joystick, a game pad, a

[0114] According to some example embodiments, a monitor 844 or other type of display device can also be connected to the system bus 808 via an interface, such as a video adapter 846. In addition to the monitor 844, the computing device 801 can include other peripheral output devices (not shown) such as speakers, printers, etc.

[0115] According to some example embodiments, the computing device 801 can operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 848. In some examples, the remote computer 848 can be a workstation, a server computer, a router, a personal computer, portable computer, microprocessor-based entertainment appliance, a peer device or other common network node, and typically includes many or all of the elements described relative to the computer, although, for purposes of brevity, only a memory / storage device 850 is illustrated. According to some example embodiments, the logical connections depicted include a wired / wireless connection to a local area network (LAN) 852 and / or larger networks, e.g., a wide area network (WAN) 854. Such LAN and WAN networking environments are commonplace in offices and companies and facilitate enterprise-wide computer networks, such as intranets, all of which can connect to a global communications network, e.g., the Internet.

[0116] In some examples when used in a LAN networking environment, the computing device 801 can be connected to the LAN 852 through a wired and / or wireless communication network interface or adapter 856. The adapter 856 can facilitate wired or wireless communication to the LAN 852, which can also include a wireless access point disposed thereon for communicating with the wireless adapter 856.

[0117] In alternative examples, when used in a WAN networking environment, the computing device 801 can include a modem 858 or can be connected to a communications server on the WAN 854, or has other mechanisms for establishing communications over the WAN 854, such as by telephone network. The modem 858, which can be internal or external and a wired or wireless device, is connected to the system bus 808 through the input device interface 842. In a networking environment, the program modules depicted as being resident on the computing device 801 can be stored on a remote memory / storage device 850. It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computing devices can be used.

[0118] According to some example embodiments, the computing device 801 can operate in the capacity of a wireless device or entity in a wireless communication network, for example a printer, a scanner, a desktop computer, and / or a portable computer, a portable data assistant, a communications satellite, any equipment or location associated with tag information that can be wirelessly sensed (e.g., kiosks, newsstands, restrooms), and a telephone. This can also include at least Wi-Fi and Bluetooth™ wireless technologies. Thus, the communication can be a predefined structure as with conventional computer networks or simply an ad hoc communication between at least two devices. TM Wireless technologies. Consequently, the communication can be a predefined structure as with conventional computer networks or simply an ad hoc communication between at least two devices.

[0119] According to some example embodiments, Wi-Fi, or Wireless Fidelity, allows connection to the Internet from a couch at home, a bed in a hotel room, or a conference room at work, without wires. To this end, Wi-Fi as referred to herein is a wireless technology similar to that used in a cell phone, which enables such devices, e.g., computers, to send and receive data indoors and out; anywhere within the range of a base station. Wi-Fi networks use radio technologies called IEEE 802.11 (a, b, g, n, etc.) to provide secure, reliable, fast wireless connectivity. Furthermore, according to some example embodiments described herein, a Wi-Fi network can be used to connect computers or the plurality of electronic devices 102-10N to each other, to the Internet and to wired networks (which use IEEE 802.3 or Ethernet). Wi-Fi networks operate in the unlicensed 2.4 and 5 GHz radio bands, at an 11 Mbps (802.11b) or 54 Mbps (802.11a) data rate, or use products that contain both bands (dual band), so the networks can provide real- world performance similar to a wire-based Fast Ethernet network. The Wi-Fi networks typically support mobile devices and can also support ad-hoc services, which are services that are not necessarily based on a connection with a base station.

[0120] Figures 9 to 11 Fig. 1 shows an operator device (102, 104, 110A-110N) and / or a device (102, 104, 110A-110N) according to an example embodiment of the present application, Figure 1 Figure 1 ​An exemplary flowchart of the operations performed by any of the servers 160. It should be understood that each block in the flowchart, and combinations of blocks in the flowchart, can be implemented by various means, such as hardware, firmware, one or more processors, circuitry, and / or other devices associated with the execution of software including one or more computer program instructions. For example, one or more of the processes described above can be embodied by computer program instructions. In this regard, the computer program instructions embodying the processes described above can be stored in the memory of a device employing an embodiment of the invention and executed by a processor in the device. It is understood that any such computer program instructions can be loaded onto a computer or other programmable device (e.g., hardware) to produce a machine such that the resulting computer or other programmable device provides an implementation of the functions specified in one or more flowchart blocks. These computer program instructions can also be stored in a non-transitory computer-readable storage memory that can instruct a computer or other programmable device to operate in a particular manner, such that the instructions stored in the computer-readable storage memory produce an article of writing whose execution can implement the functions specified in one or more flowchart blocks. Computer program instructions can also be loaded onto a computer or other programmable device to cause a series of operations to be performed on the computer or other programmable device, thereby producing a computer-implemented method, such that the instructions, which execute on the computer or other programmable device, provide operations for implementing the functions specified in one or more flowchart blocks. Therefore, Figures 9 to 11 When executed, the operation transforms the computer or processing circuitry into a specific machine configured to perform exemplary embodiments of the present invention. Therefore, Figures 9 to 11 The operation definition is used to configure a computer or processor to execute the algorithm of the exemplary implementation. In some cases, an instance of a processor may be provided for a general-purpose computer, which executes... Figures 9 to 11 The algorithm transforms a general-purpose computer into a specific machine configured to execute an exemplary implementation.

[0121] Therefore, the boxes in a flowchart support combinations of devices for performing a specified function and combinations of operations for performing the specified function. It will also be understood that one or more boxes in a flowchart, as well as combinations of boxes in a flowchart, can be implemented by a hardware-based dedicated computer system or a combination of dedicated hardware and computer instructions to perform the specified function.

[0122] Figure 9 A flowchart of a method 900 for improving the performance of performing workflow operations, according to an exemplary embodiment, is shown.

[0123] The method 900 begins at step 902. At step 904, any of the operator devices 110A-110N and / or the server 160 can include a means, such as the communication module 240, to receive a voice dialogue associated with a task of a workflow (e.g., but not limited to, an item pick workflow, a cycle count workflow, etc.). The voice dialogue can include a first voice prompt and a first voice response. The operator can provide the first voice response in response to the first voice prompt. As described earlier, the voice dialogue can represent a sequence of (a) voice prompts that can be generated to provide instructions to the operator, and (b) voice responses that can be provided by the operator to the respective voice prompts. In other words, the voice prompts can include machine-generated instructions (e.g., verbal instructions, voice commands, visual instructions, etc.) that can be provided to the operator on the operator devices 110A-110N, which the respective operator can use to perform one or more tasks of the workflow. Further, the voice responses can include responses provided by the operator in response to the respective voice prompts.

[0124] At step 906, any of the operator devices 110A-110N and / or the server 160 can include a means, such as the processor 210, to identify a performance state associated with the performance of the workflow task. According to an example embodiment, the performance state can be identified prior to a second voice prompt after the first voice prompt is provided. In other words, the performance state is identified in near real-time within a certain time of providing the first voice response to the first voice prompt and prior to providing the second voice prompt to the operator. The processor can identify the performance state based on analyzing the voice dialogue using a machine learning model, as previously described with reference to FIGS. 1-3. Figure 4 Further details of the identification of the performance state are described with reference to FIGS. 4-6. Figure 10 Further details of the identification of the performance state are described with reference to FIGS. 4-6.

[0125] According to some example embodiments described herein, the performance state can indicate a progress and / or performance related to the performance of the workflow task. For example, in one example, the performance state can indicate whether the task and / or the workflow can be performed within an expected time or will be delayed / incomplete within the expected time. Alternatively and / or additionally, in another example, the performance state can indicate whether the task and / or the workflow can be performed with an expected quality. According to some example embodiments, the performance state can indicate at least one of (a) an expected delay in the performance of the workflow task, (b) an incorrect performance of the workflow task, and / or (c) a coordination gap corresponding to the first voice prompt and the first voice response. In some examples, the coordination gap corresponding to the first voice prompt and the first voice response can be due to an unexpected response to the voice prompt provided by the operator. In other words, the coordination gap can indicate a situation in which the voice response provided by the operator is inconsistent with an expected voice response.

[0126] For example, a task of a workflow can correspond to an item pick operation, in which an operator must pick an item (e.g., a package, a shipment, a product, etc.) from various storage locations within an industrial environment (e.g., a warehouse or inventory). In such a scenario, a performance state based on speech dialogue recognition can indicate a slow pick or a normal pick, an execution of the pick task performed by the operator. For example, if the time taken to pick the item is slower than an expected time to pick the item, the performance state can indicate a slow pick performed by the operator. Thus, in a case where the time taken to pick the item is in accordance with the expected time to pick the item, the performance state based on speech dialogue recognition can indicate a normal pick execution of the pick task. In this regard, during the execution process of the pick workflow activity, the performance state in this scenario (i.e., slow pick and / or normal pick) can be identified in near real-time, i.e., based on the initial exchange of speech prompts and / or speech responses related to the pick workflow activity and prior to completion of the execution of the pick workflow activity.

[0127] At step 908, any of the operator devices 110A-110N and / or the server 160 can include a device, such as the processor 210, to generate a message indicating a suggestion to improve the performance state of the task. According to some example embodiments, the message can be generated prior to providing the second speech prompt to the operator. According to example embodiments, the message can be generated in the form of the speech prompt itself, followed by the generation of the second speech prompt to the operator associated with the next instruction. In another example embodiment, the message can be generated as a visual instruction on a display screen of an electronic device (e.g., the operator device 102, 110A-110N). In some example embodiments, the message indicating a suggestion to improve the performance state can be provided after completion of the execution of the workflow task (e.g., but not limited to, provided to the operator by a supervisor in an offline manner).

[0128] According to some example embodiments, the suggestion provided at step 908 can correspond to an activity that can be performed by the operator and / or a supervisor to improve the performance state of the execution of the workflow task. For example, the suggestion can indicate an activity that can be performed by the operator to cause the workflow task to be performed effectively or faster. For example, in one example, the suggestion can indicate an option to reassign the workflow task from the operator to another operator (i.e., an operator that can perform the task more proficiently). In another example, in a case where the task is a pick workflow for picking an item, the suggestion can indicate a correct location for picking the item. Similarly, in another example, the suggestion can indicate a correct destination for placing the item. In some example embodiments, the suggestion can also indicate a route for reaching the item pick location or the item place destination.

[0129] In some other example embodiments, the suggestion can indicate an activity that can be performed to improve the voice recognition of the voice responses provided by the respective operator on the operator device (102, 104, 110A-110N) used by the operator. For example, in one example, the suggestion can indicate that the voice recognition model used by the operator device (102, 104, 110A-110N) is required to be retrained to recognize the voice responses provided by the operator. It will be appreciated that in some example cases, the voice responses provided by the operator can not be recognized by the operator device (102, 104, 110A-110N) due to the presence of background noise in the work environment, thereby affecting the performance state (e.g., resulting in delayed performance of the task). In this regard, in one example, the suggestion can indicate that the voice recognition model is to be retrained according to the noise level in the work environment. Further, in another example embodiment, the suggestion can indicate that the voice recognition model is to be retrained according to the speaker-dependent voice template of the operator performing the workflow task. Further, in some example embodiments, the suggestion can indicate that a new task or a new workflow can be preferable for the operator (e.g., according to the skill level or ability of the operator).

[0130] Thus, by implementation of the various example embodiments described herein, the operator device (102, 104, 110A-110N) can include a means such as the processor 210 to: (a) identify the performance state in near real-time (i.e., during performance of the workflow task) based on an analysis of the ongoing voice conversation (i.e., the current voice prompt and the current voice response), and (b) provide a suggestion that can improve the performance state of the workflow. In this regard, in some examples, the suggestion can be provided "on the fly" (i.e., during performance of the workflow task). Alternatively and / or additionally, in some examples, the suggestion can be provided offline (e.g., by a supervisor) after the operator has performed the tasks of the workflow. The method 900 stops at step 910.

[0131] Figure 10 A flowchart representing a method 1000 for identifying a performance state associated with performance of a workflow operation is shown, in accordance with an example embodiment. In some example embodiments, the method 1000 can be performed by any of the devices (102, 104, 110A-110N, 150, 160) as previously described with reference to Figure 1 The method 1000 can be performed by the workflow system 400, in accordance with some example embodiments, as described with reference to Figure 4 The method 1000 starts at step 1002.

[0132] According to some example embodiments, the method 1000 can begin in response to initiating execution of a workflow task. For example, as shown by the connector labeled "A", in some example embodiments, the method 1000 can begin in response to receiving a voice dialog including at least a first voice prompt associated with a workflow task and a first voice response. In other words, the method 1000 can begin after an initial few exchanges of voice prompts and voice responses when initiating execution of a workflow task. Alternatively and / or additionally, in some example embodiments, the method 1000 can begin when executing a task pre-defined step of a workflow. For example, in example embodiments, the method 1000 can begin in response to identifying a "trigger phrase" that can initiate the method 1000. The trigger phrase can be defined based on a context associated with a workflow task. For example, the trigger phrase can be "ready" that can be provided as a voice response by an operator. In another example, the trigger phrase would be "position" that can be provided as a voice prompt to an operator. Thus, various trigger phrases can be used to initiate the method 1000 for identifying a performance state related to execution of a workflow.

[0133] At step 1002, the workflow system 400 can include a means, such as the communication interface 440, to access (e.g., via the workflow database 300) a data file including a historical voice dialog corresponding to a past execution of a workflow task. In other words, to perform identification of a performance state associated with an ongoing task of a workflow, the workflow system 400 can access a voice dialog associated with a historical execution or a previous execution of a task (e.g., a same type of task) of a workflow. Details are described later with reference to Figure 12 Examples of data files including a historical voice dialog or a previous voice dialog are shown and described.

[0134] At step 1004, the workflow system 400 can include a means, such as the context identification module 420, to parse the data file including the historical voice dialog to identify a context key phrase. Further, in some example embodiments, the context identification module 420 can also identify a context parameter associated with the context key phrase based on the parsing of the data file. According to some example embodiments, the context key phrase can be identified based on a pre-defined rule by parsing the historical voice dialog. In some examples, the parsing of the historical voice dialog can also include performing morphological disambiguation and string processing on the historical voice dialog, and extracting a feature set from the historical voice dialog. The feature set can include instances of locations where the context key phrase can exist. Details are described earlier with reference to Figure 4 Details of identification of a context key phrase and / or a context parameter associated with the context key phrase are described.

[0135] At step 1006, the contextual keywords and the context parameters associated with the contextual keywords can be provided as input features to train a machine learning model. The machine learning model can be used to identify a performance state associated with the execution of the workflow task. Moving to step 1008, the workflow system 400 can include a device such as the machine learning engine 430 that can use the machine learning model to identify a pattern associated with the contextual keywords and / or the context parameters. The identification of the pattern associated with the contextual keywords and / or the context parameters can be performed in a similar manner as previously described with reference to Figure 4 the pattern associated with the contextual keywords and / or the context parameters.

[0136] In some examples, the machine learning engine 430 can identify the pattern associated with the contextual keywords and the context parameters based on predefined rules. In some examples, the predefined rules can be defined according to the type of the workflow task. In other words, there can be different rules for identifying the pattern according to different types of tasks and / or workflow operations. To this end, it should be appreciated that there can be various tasks performed for different types of workflow execution. For example, in a materials handling environment, there can be different types for a workflow such as, but not limited to, an item pick workflow, an item place workflow, a stock replenishment workflow, a cycle count workflow, and the like that can be performed by various operators. Thus, the predefined rules for identifying the pattern can vary depending on the type of the workflow.

[0137] At step 1010, the workflow system 400 can include a device such as the machine learning engine 430 to predict a classification indicative of a performance state associated with the execution of the workflow task. As previously described, the performance state can be indicative of the progress of the execution of the workflow task. For example, the performance state can be indicative of a normal execution of the workflow task. In another example, the performance state can be indicative of a delayed execution of the workflow task. In another example, the performance state can be indicative of a fast execution of the workflow task. In another example, the performance state can be indicative of an incorrect execution of the workflow task. The method 1000 stops at step 1012.

[0138] Figure 11 A flow chart representing a method 1100 for providing suggestions to improve performance of a worker in performing a workflow operation is shown in accordance with an example embodiment. In some example embodiments, the method 1000 can be performed by any of the devices (102, 104, 110A-110N, 150, 160) as previously described with reference to Figure 1 the method 1100 starts at step 1102.

[0139] At step 1104, the workflow execution system 100 can include a device, such as a voice control device (e.g., the voice control device 104), to provide a first voice prompt associated with the workflow task. In this regard, as previously noted, the voice prompts referenced herein can represent audio / voice based instructions that can be provided to an operator to perform a step associated with the workflow task.

[0140] For example, the workflow operation can relate to the picking of various items in a warehouse. In this regard, the task of the workflow can be to pick an item from a storage location in the warehouse. Accordingly, in such an example, the voice control device 104 can provide a first voice prompt that instructs an operator in the warehouse to a location from which to pick the item. In this regard, when the location is reached, the operator can provide a voice response that indicates that the operator has reached the location. Further, subsequent voice prompts related to picking other items can be provided by the voice control device 104.

[0141] At step 1106, the voice control device 104 can receive a first voice response in response to the first voice prompt. As previously noted, the voice responses referenced herein can represent verbal input that is replied by an operator in response to the instructions provided in the voice prompts. For example, for a picking operation, the first voice response can indicate a confirmation provided by the operator when the location is reached. In one example, the first voice response can indicate a number and / or letter of a check digit that can be used to uniquely identify the storage location. Accordingly, in accordance with the various example embodiments described herein, the voice control device 104 can receive a voice response in response to each voice prompt provided to an operator, respectively.

[0142] Moving to step 1108, the voice control device 104 can include a device, such as the processor 210, that can utilize the machine learning engine 430 to predict a performance state associated with the performance of the workflow task. As previously noted with reference to Figure 4 the machine learning engine 430 can predict the performance state based on analyzing the first voice prompt and the first voice response. In this regard, the first voice prompt and the first voice response can be analyzed by using patterns associated with historical voice conversations, as previously noted with reference to Figures 1 to 10 the machine learning engine 430.

[0143] At step 1110, the processor 210 can determine whether the performance state predicted at step 1108 indicates normal operation. In this regard, normal operation can indicate that the operator performed the workflow task in accordance with an expected time or quality metric associated with the type of workflow. According to some example embodiments, the performance state can indicate normal operation based on a comparison of the performance state predicted at step 1108 to a predefined threshold. For example, in the example of a pick workflow operation, if the pick operation is a delayed pick (i.e., the time taken by the operator to perform the task of the pick operation exceeds the expected time), the performance state can be determined to not indicate normal operation. Similarly, for a pick workflow operation, if the time taken by the operator to perform a step of the pick operation is less than or equal to the expected time for performing that step, the performance state can be determined to be normal operation.

[0144] In response to determining at step 1110 that the performance state indicates normal operation, the method moves to step 1116. Alternatively, if the performance state does not indicate normal operation, the method moves to step 1112.

[0145] At step 1112, the processor 210 can identify an issue associated with the performance of the workflow task. In example embodiments, the processor can utilize the workflow system 400 as previously described to parse a data file corresponding to historical data, i.e., a previous exchange of voice prompts and voice responses (e.g., associated with a task of the same type of workflow) to further identify patterns using the first voice prompt, the first voice response, and the historical data to identify an issue associated with the performance of the workflow task.

[0146] In one example, the issue can indicate a coordination gap in the voice dialogue, i.e., an exchange of voice-based messages between the operator and the voice-controlled device 104. For example, the coordination gap can indicate a mismatch between a voice response expected by the voice-controlled device 104 and an actual voice response provided by the operator in response to the voice prompt. In other words, in one example, the issue can indicate an unexpected or incorrect voice response provided by the operator to the voice-controlled device 104. In another example, the issue can be related to a failure or error in speech recognition performed by the voice-controlled device 104. This can occur in instances in which the speech recognition model used by the voice-controlled device 104 is trained using a speaker-dependent template that does not include a voice template of the current operator performing the execution of the workflow task. Other types of issues (e.g., battery interruption, device malfunction, incorrect route used by the operator, etc.) can be possible, which can cause the performance state of the task execution to deviate from normal operation.

[0147] At step 1114, the processor 210 can generate a message indicating a suggestion to improve the performance state of the task. As previously referenced with respect toFigures 1 to 10 As described, according to some examples, the suggestion corresponds to an activity that can be performed by the operator and / or supervisor to improve a performance state of the execution of the workflow task. In other words, in some examples, the suggestion can indicate an activity that can be performed by the operator to cause the workflow task to be executed effectively or more quickly. According to the example embodiments described herein, the suggestion can be generated as a voice prompt that can be output by the speaker of the voice-controlled device 104 and / or a visual instruction that can be displayed on the display screen of the electronic device 102. As Figures 1 to 10 As described, there can be different types of suggestions (e.g., reassign the workflow to another worker, retrain the voice recognition model, provide an expected voice response, etc.) that can be generated and provided to the operator for improving the performance state.

[0148] Accordingly, by implementation of the example embodiments described herein, in some cases, where the performance state is determined to indicate execution of the workflow task that is not in accordance with normal operation, at step 1110, a suggestion can be provided to the operator. Alternatively, where the performance state is indicative of normal operation, the method 1100 can move to step 1116, where a second voice prompt following the first voice prompt in the workflow task can be provided to the operator by the voice-controlled device 104. The second voice prompt can include instructions for a next step that is to be performed by the operator following completion of the instructions provided in the first prompt. Accordingly, as shown at step 1118, the voice-controlled device 104 can receive a second voice response in response to the second voice prompt.

[0149] As such, where the performance state is indicative of normal operation, the voice conversation (i.e., the instructions provided by the voice-controlled device 104 in the form of voice prompts and the voice responses received to the voice prompts) can continue until the task or activity of the workflow operation is completed. Further, as the operator completes the required steps of the task, at step 1120, the voice-controlled device 104 can provide a voice prompt (e.g., a third voice prompt) that can indicate completion of the workflow task. The method stops at step 1122.

[0150] Figure 12 An example scenario depicting a workflow operation that can be performed by an operator is shown in accordance with example embodiments. As shown, a work environment 1200 can include a server 160 (e.g., a warehouse management system, WMS), a voice-controlled device 104, and an electronic device 102. The server 160, the voice-controlled device 104, and the electronic device 102 can be communicatively coupled to one another over the network 103. As previously described with reference to Figure 1 As described, the electronic device 102 and the voice-controlled device 104 can correspond to devices that can be used by an operator to perform a workflow operation (e.g., operator devices 110A-110N). As described, the electronic device 102 and the voice-controlled device 104 can correspond to devices that can be used by an operator to perform a workflow operation (e.g., operator devices 110A-110N).

[0151] According to example embodiments, an operator can use the voice controlled device 104 and the electronic device 102 to perform one or more tasks of a workflow operation. In one example, the workflow operation can be an item pick operation for picking one or more items, for example, from a storage location in a warehouse. In this regard, in example embodiments, the operator can wear the voice controlled device 104 (e.g., a headset device) and receive instructions from the electronic device 102 in the form of voice prompts to perform various steps associated with the workflow operation. For example, the operator can receive one or more voice prompts on the voice controlled device 104 that can include instructions related to the picking of an item, such as instructions for reaching a storage location, identifying an item to pick, confirming the item for picking, etc. In other words, the various steps of the workflow operation can be performed based on a voice conversation (i.e., an exchange of voice prompts and voice responses) between the operator and the operator device.

[0152] For example, as shown in Figure 12 the voice conversation 1202 can include a plurality of voice prompts that can be provided by the voice controlled device 104 (referred to herein as Talkman) to the operator for performing steps of a workflow operation. Illustratively, the voice prompts are "One line, H U color is purple, check digit middle," "bravo golf One Six alpha Zero Two," etc. In response to each voice prompt, the operator can perform the tasks required at the respective step and provide a voice response to the voice prompt. For example, as shown in the voice conversation 1202, the voice responses provided by the operator include "position," "ready," "8," "4," etc. According to some examples, the voice responses can indicate the performance or non-performance of the tasks directed in the voice prompts. In some examples, the voice responses can indicate any of the following: a confirmation indicating the performance of the step of the workflow task, a data value, etc.

[0153] Further, the voice conversation can be stored in one or more data files. As shown, the server 160 can receive a data file 1206 corresponding to the exchange of voice prompts and voice responses between the voice controlled device 104 and the operator. To this end, the server 160 can store the data file 1206 in a database (e.g., the workflow database 300, as shown in Figure 3 Further, according to the example embodiments, the workflow system 400 can use the data file 1206 to identify a performance state associated with the performance of the workflow task.

[0154] Additionally, as shown in the figure, electronic device 102 can provide data messages to server 160 and receive one or more host responses from server 160. In some examples, data messages may include, for example, but not limited to, information relating to voice dialogue (i.e., the exchange of voice prompts and voice responses) for various tasks of a workflow performed using electronic device 102, requests to download files including workflows and workflow-related instructions from server 160, configuration files for access control or role-based configuration of electronic device 102 based on the operator.

[0155] According to an exemplary embodiment, when initiating the execution of a workflow operation, the electronic device 102 and the voice control device 104 can maintain the workflow state 1203 and the operator state 1204, such as Figure 12 As shown. Workflow state 1203 and operator state 1204 can represent the application state of a workflow step that may be currently being executed. According to an exemplary embodiment, the workflow state and operator state can be synchronized with each other in order to efficiently execute each step of the workflow. In other words, during the effective execution of the workflow, the coordination between the voice prompts provided by the voice control device 104 and the voice responses provided by the operator will ensure that the state change from the current state to the next state of each of the workflow state and operator state should occur simultaneously. In this respect, the states of the workflow state and operator state can change when each step of the workflow is executed. The following paragraphs describe examples of the state changes of the workflow state and operator state when each step of the workflow operation is executed.

[0156] In one example, for an item picking operation workflow, as shown in the figure, workflow state 1203 and operator state 1204 can be "Order Information," "Location Information + Type," "Location Verification," "Material Information," etc. In this regard, when initiating the execution of the item picking operation workflow, a first voice prompt associated with the order information can be provided to the operator from the voice control device 104. Therefore, in this case, the workflow state can be "Order Information." Furthermore, in response to the first voice prompt, the operator can provide a first voice response instructing the operator to confirm the order information. In this case, the operator state can also be "Order Information." Additionally, a second voice prompt associated with the location and item type information used to pick up the item can be provided to the operator. This can move the workflow state to "Location Information + Type." Furthermore, when the operator arrives at the location used to pick up the item, the operator can provide a second voice response indicating verification or confirmation that the operator is in the desired location. At this stage, the operator state can be "Location Information + Type." Therefore, in a similar manner, the workflow state and operator state can be changed as each step of the workflow is executed.

[0157] According to various example embodiments described herein, in some cases, there can be a coordination gap between the workflow state and the operator state during the performance of a task of a workflow. For example, with reference to the voice dialog 1202, it can be observed that similar voice prompts (e.g., “One line, H U color is purple, check digit middle,” “bravo golf One Six alpha Zero Two”) are repeatedly provided to the operator by the voice control device 104 (e.g., Talkman). In some examples, this can occur because the operator can not provide the expected voice response to the voice control device 104. In some examples, the operator can provide the expected voice response, however, the voice recognition engine of the voice control device 104 can not recognize the voice response provided by the operator. In this regard, as previously described with reference to Figures 1 to 11 the performance state related to the performance of a task of a workflow can be identified in near real-time for identifying such issues (e.g., the coordination gap between the workflow state 1203 and the operator state 1204). In other words, the performance state can be identified to indicate whether the performance of the task of the workflow is proceeding normally. Further, as previously described with reference to Figures 1 to 11 the context suggestions can be provided to improve the performance state of the task.

[0158] As shown, for the item pick-up workflow operation, in example embodiments, the server 160 can include a device such as a processing unit to analyze the voice dialog 1202 and determine (1208) whether the pick-up operation is performed as a normal pick-up or a slow pick-up. In this regard, in the case where the pick-up operation is performed as a normal pick-up, the server 160 can instruct the electronic device 102 and the voice control device 104 to continue performing the task of the workflow and move to the next step. Alternatively, as previously described, context suggestions can be provided to the operator. The context suggestions can be received at the voice control device 104 and / or the electronic device 102. Thus, through the implementation of the various example embodiments described herein, a performance state indicating the progress and / or performance of the performance of the task of the workflow operation can be identified. Further, based on the performance state, context suggestions for improving the performance state can be provided. This will result in improved performance of the workflow operation, thereby increasing the productivity of the operator and the overall throughput of the work environment.

[0159] In some example embodiments, some of the operations herein can be modified or further amplified as described below. Further, in some embodiments, additional optional operations can also be included. It should be understood that each of the modifications, optional additions, or amplifications described herein can be included in the operations herein, either alone or in combination with any others of the features described herein.

[0160] The foregoing method descriptions and the process flow diagrams are provided merely as illustrative examples and are not intended to require or imply that the steps of the various embodiments must be performed in the order presented. As will be appreciated by one of ordinary skill in the art, the order of steps in the foregoing embodiments can be performed in any order. Words such as "thereafter," "then," "next," etc. are not intended to limit the order of the steps; these words are simply used to guide the reader through the description of the methods. Further, any reference to claim elements in the singular, for example, using the articles "one," "a" or "an," is not

[0161] The various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

[0162] The hardware used to implement various illustrative logics, logical blocks, modules, and circuits described in connection with the aspects disclosed herein can be implemented or performed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field

[0163] In one or more exemplary aspects, the functions described can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions can be stored on or transmitted over as one or more instructions or code on a non-transitory computer-readable medium or a non-transitory processor-readable medium. The steps of a method or algorithm disclosed herein can be embodied in a processor-executable software module (or processor-executable software instructions) which can reside on a non-transitory computer- or processor-readable storage medium. Non-transitory computer- or processor-readable storage media can be any storage media that can be accessed by a computer or a processor. By way of example but not limitation, such non-transitory computer- or processor-readable media can include RAM, ROM, EEPROM, FLASH memory, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Disk and disc, as used herein, includes compact discs (CD), laser discs, optical discs, digital versatile discs (DVD), floppy disks, and Blu-ray discs where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above are also included within the scope of non-transitory computer- or processor-readable media. Additionally, the operations of a method or algorithm can reside in one or any combination of the above memory hardware, which can be incorporated into a computer program product.

[0164] While various embodiments in accordance with the principles disclosed herein have been shown and described, modifications thereof can be made by one skilled in the art without departing from the spirit and teachings of the disclosure. The embodiments described herein are representative only and are not intended as limitations on the scope of the disclosure. Many variations, combinations, and modifications are possible and are within the scope of the disclosure. Alternative embodiments incorporating one or more aspects of the disclosure can be made without departing from the scope of the disclosure. Accordingly, the scope of protection is not limited to the description set out above, but is given by the appended claims, and their equivalents, to the fullest scope permissible. Each claim is incorporated into the specification as further disclosure separately, and each claim is a separate embodiment of one or more of the applications. Additionally, any combination of the recited advantages and features can be used in any combination, but not all of the advantages and features are required in any particular embodiment. The application is not limited to the details given above, but can be practiced with modifications and alterations within the scope and spirit of the claims.

[0165] Furthermore, the section headings used herein are for organizational purposes only and are not meant to be used as limiting in construing the contents of the disclosure. As used herein, the article "a" is intended to include one or more items. Moreover, the use of the term "about" is intended to allow for variations, such as due to reasonable expected variations in temperature, pressure, and other conditions, reagents, and materials. Additionally, the use of the term "about" is intended to cover variations that are within the scope of what would be expected by a person of ordinary skill in the art to be within the scope of the disclosure. Furthermore, the section headings used herein are presented for consistency with the suggestions of 37 C.F.R. 1.77 and to provide organizational cues. These headings shall not limit or characterize the inventiveness of any subject matter claimed in any claims that can be presented in this disclosure. For example, the description of technology in the “BACKGROUND” section is not to be interpreted as an admission that technology is prior art to any subject matter disclosed herein. The “SUMMARY” is not to be interpreted as an identification of one or more claimed applications that are described in the “SUMMARY.” The “SUMMARY” can contain additional subject matter that is not part of the claimed applications. Additionally, any references in the “SUMMARY” to “application” should not be interpreted as a limitation on the scope of the disclosure. Furthermore, any reference in the “SUMMARY” to “invention” should not be interpreted as a limitation on the scope of the disclosure. In all instances, the scope of the claims shall be considered on their own merits in light of this disclosure, and not with reference to steps of the methods of operation outlined in any section of this disclosure, unless the steps are expressly claimed. The foregoing description of certain implementations will be better understood when read in conjunction with the accompanying drawings, in which:

[0166] Further, techniques, systems, subsystems, and methods described and illustrated in the various embodiments as discrete or separate can be combined or integrated with other systems, modules, techniques, or methods without departing from the scope of the disclosure. Other items shown or discussed as separate from the other items or implementing the other items can be implemented jointly with the other items or can be separated into separate items. The implementations described herein can be implemented in electronic hardware, computer software, or any combination thereof. Any features described as devices, systems, or methods can be implemented in hardware, software, firmware, or any combination thereof. Although various implementations can have often been described with reference to particular implementations, it is to be understood that variations and modifications can be affected when the applications are constructed in accordance with the principles of the present disclosure. Accordingly, it is intended that the applications be construed broadly and interpreted from the claims in accordance with the full scope of equivalents, analogies, and equivalents thereof.

[0167] Many modifications and other implementations of the applications set forth herein will occur to those skilled in the art upon reading of the foregoing description and studying the drawings. The order in which steps are presented in the methods described herein is not meant to be limiting, and the steps can be performed in any order, or concurrently, or in some instances, not performed at all. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation. Numerous additional modifications will be apparent to those skilled in the art in view of the foregoing description and drawings. Thus, various implementations of the applications have been described. It will be understood that the applications are not limited to the particular implementations described but are capable of a variety of modifications and alternative constructions. Accordingly, the applications are not limited to the details discussed above, but rather are capable of a variety of modifications and alternative constructions as would be understood by persons of ordinary skill in the art upon reading the foregoing description. Although the foregoing application has been described in some detail for the purposes of clarity and understanding, it will be appreciated that certain changes and modifications can be practiced within the scope of the appended claims. Accordingly, the application is not to be limited to the exact details shown and described, for it is desired that only such limitations be placed on the application as described in the appended claims. All patents and publications mentioned herein are hereby incorporated by reference in their entirety for the teachings relevant to the sentence and / or paragraph in which the reference is presented.

[0168] It should be appreciated that any such computer program instructions and / or other type of code can be loaded onto a computer, a processor or other programmable apparatus a circuit to produce a machine, such that the computer, processor, other programmable circuit, which executes the code on the machine becomes an apparatus for practicing the various functions described herein.

[0169] It should also be noted that all or some of the information presented by the example displays discussed herein can be based on data received, generated, and / or maintained by one or more components of the local or networked system and / or circuit 200. In some embodiments, one or more external systems, such as remote cloud computing and / or data storage systems, can also be utilized to provide at least some of the functionality discussed herein.

[0170] As described above and should be appreciated based on the disclosure, embodiments of the application can be configured as methods, personal computers, servers, mobile devices, backend network devices, etc. Accordingly, embodiments can include various apparatuses including entirely hardware or any combination of software and hardware. Moreover, embodiments can take the form of a computer program product on at least one non-transitory computer-readable storage medium having computer-readable program instructions (e.g., computer software) embodied in the storage medium. Any suitable computer-readable storage medium can be utilized, including a non-transitory hard disk, a CD-ROM, a flash memory, an optical storage device, or a magnetic storage device.

[0171] Embodiments of the application have been described above with reference to block diagrams and flowchart illustrations of the methods, apparatuses, systems, and computer program products. It is to be understood that each block of the circuit diagrams and flowchart illustrations, and combinations of blocks in the circuit diagrams and flowchart illustrations, can be implemented by various means, including computer program instructions in one or more computer-readable storage media (or machine-readable media) referenced above. These computer program instructions can be used to cause one or more computers, special-purpose computer-readable storage media (or machine-readable storage media) referenced above. Figure 2 The processor 210, workflow database 300, and / or workflow system 400 discussed above, to produce a machine, such that the computer program product includes instructions that are executed on a computer or other programmable data processing apparatus to produce a machine, so that the computer program product includes instructions executed on the computer or other programmable data processing apparatus to produce a machine.

[0172] These computer program instructions can also be stored in a computer- readable memory (e.g., memory 220) that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including computer-readable instructions for implementing the functionality discussed herein. The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions that are executed on the computer or other programmable apparatus provide steps for implementing the functionality discussed herein.

[0173] Accordingly, the blocks of the flowchart illustrations support combinations of means for performing the specified functions, combinations of steps for performing the specified functions and program instruction means for performing the specified functions. It will also be understood that each block of the circuit diagram and process flowchart illustrations, and combinations of blocks in the circuit diagram and process flowchart illustrations, can be implemented by special purpose hardware-based computer systems that perform the specified functions or steps, or combinations of special purpose hardware and computer instructions.

Claims

1. A voice control device, the voice control device comprising: a microphone; a speaker; a processor communicatively coupled to at least one of the microphone and the speaker, wherein the processor is configured to: generate, via the speaker, a voice prompt associated with a task of a workflow; identify, via the microphone, a voice response from a worker in response to the voice prompt, wherein the voice prompt and the voice response comprise a voice conversation; change, from the voice conversation: a workflow state from a first workflow state to a second workflow state, and an operator state from a first operator state to a second operator state; identify, based on analyzing the voice conversation using a machine learning model, a performance state associated with performance of the task of the workflow prior to a next voice prompt after providing the voice prompt, wherein the performance state indicates at least a coordination gap between the first workflow state and the first operator state; and generate a message comprising a suggestion to improve the performance state of the task, wherein the processor is configured to: access a data file comprising historical voice conversations corresponding to past performed tasks; parse the data file to identify, from a plurality of historical voice responses in the historical voice conversations, a contextual keyword and a contextual parameter associated with the contextual keyword, wherein the contextual parameter comprises at least one of: a frequency of occurrence of the contextual keyword in the historical voice conversations, and timing information corresponding to each occurrence of the contextual keyword in the historical voice conversations; and provide the contextual keyword and the contextual parameter as input features to train the machine learning model; identify, using the machine learning model, a pattern associated with the contextual keyword and the contextual parameter, wherein the machine learning model identifies the pattern based on predefined rules defined according to a task type of the workflow; and predict, by using the pattern, a classification indicating the performance state associated with performing a task.

2. The voice control device of claim 1, wherein the performance state indicates at least one of: a delay in performance of a task of the workflow; and an incorrect performance of the task of the workflow.

3. The voice control device of claim 1, wherein the suggestion indicates at least one of: a correct location for picking up an item; a correct destination for placing the item; a route to one of: an item pick-up location or an item drop-off destination; an expected voice response to the voice prompt according to the workflow; retraining a speech recognition model according to a noise level in a work environment; retraining the speech recognition model according to a speaker-dependent voice template; a new workflow according to the worker skill level or ability; and an option to reassign the task of the worker to another worker.

4. A method for voice control, the method comprising: generating, via a speaker, a voice prompt associated with a task of a workflow; identifying, via a microphone, a voice response from a worker in response to the voice prompt, wherein the voice prompt and the voice response comprise a voice conversation; changing, from the voice conversation: a workflow state from a first workflow state to a second workflow state, and an operator state from a first operator state to a second operator state; identifying, based on analyzing the voice conversation using a machine learning model, a performance state associated with performance of the task of the workflow prior to a next voice prompt after providing the voice prompt, wherein the performance state indicates at least a coordination gap between the first workflow state and the first operator state; and generating a message comprising a suggestion to improve the performance state of the task, wherein the method further comprises: accessing a data file comprising historical voice conversations corresponding to past performed tasks; parsing the data file to identify, from a plurality of historical voice responses in the historical voice conversations, a contextual keyword and a contextual parameter associated with the contextual keyword, wherein the contextual parameter comprises at least one of: a frequency of occurrence of the contextual keyword in the historical voice conversations, and timing information corresponding to each occurrence of the contextual keyword in the historical voice conversations; and providing the contextual keyword and the contextual parameter as input features to train the machine learning model; identifying, using the machine learning model, a pattern associated with the contextual keyword and the contextual parameter, wherein the machine learning model identifies the pattern based on predefined rules defined according to a task type of the workflow; and predicting, by using the pattern, a classification indicating the performance state associated with performing a task.

4. The voice control device of claim 1, wherein the task is to pick up an item, and wherein the processor is configured to identify the performance state indicating one of: a slow pick up, in which a time taken to pick up the item is slower than an expected time to pick up the item; and a normal pick up, in which the time taken to pick up the item is consistent with the expected time to pick up the item.

5. A workflow execution system, comprising: a processor configured to: receive a voice conversation associated with a task of a workflow to be executed on a voice control device, the voice conversation including a first voice prompt and a first voice response to the first voice prompt; change, from the voice conversation: a workflow state from a first workflow state to a second workflow state, and an operator state from a first operator state to a second operator state; identify, prior to a second voice prompt provided after the first voice prompt, a performance state associated with execution of the task of the workflow based on analyzing the voice conversation using a machine learning model, wherein the performance state indicates at least a coordination gap between the second workflow state and the second operator state; and generate a message indicating a suggestion to improve the performance state of the task, wherein the processor is configured to: access a data file including historical voice conversations corresponding to past executions of the task of the workflow; parse the data file to identify, from a plurality of historical voice responses in the historical voice conversations, contextual keywords and contextual parameters associated with the contextual keywords, wherein the contextual parameters include at least one of: a frequency of occurrence of the contextual keywords in the historical voice conversations, and timing information corresponding to each occurrence of the contextual keywords in the historical voice conversations; and provide the contextual keywords and the contextual parameters as input features to train the machine learning model; identify, using the machine learning model, a pattern associated with the contextual keywords and the contextual parameters, wherein the machine learning model identifies the pattern based on predefined rules defined according to a task type of the workflow; and predict, by using the pattern, a classification indicating the performance state associated with executing the task.

6. The workflow execution system of claim 5, wherein the performance state indicates at least one of: a delay in execution of the task of the workflow; and an incorrect execution of the task of the workflow.

Images