Apparatus, computer program product and computer-implemented method for adapting speech recognition based on expected response

Abstract

Embodiments of the present disclosure provide for adapting a speech recognition confidence score to one or more users based on an expected response. Some embodiments receive input speech and generate a recognition hypothesis based on the input speech and using a recognition algorithm. The recognition hypothesis may include at least one pair including a word and an initial confidence score associated with the word. Some embodiments determine whether the identification hypothesis matches an expected response based at least in part on comparing the word to a corresponding word in the expected response. Some embodiments update statistics associated with the word in response to determining that the recognition hypothesis matches the expected response.

Description

Technical Field

[0001] Embodiments of the present disclosure relate generally to speech recognition, and particularly to adapting a speech recognition confidence score to a user using expected responses. Background Art

[0002] Speech recognition has simplified many tasks in various contexts, for example, in the workplace. However, applicants have identified problems with current implementations of speech recognition. Through applied effort, ingenuity, and innovation, applicants have addressed many of these identified problems by developing embodiments of the present disclosure, which are described in detail below. Summary of the Invention

[0003] In general, embodiments of the present disclosure herein provide for adapting confidence scores for speech recognition to users using expected responses. Other implementations for adapting confidence scores will be or will become apparent to those skilled in the art upon examination of the following figures and detailed description. It is intended that all such additional implementations included within this description be within the scope of this disclosure and protected by the following claims.

[0004] According to one aspect of the present disclosure, a computer-implemented method for adapting a speech recognition confidence score to a user is provided. The computer-implemented method can be performed using any computing device and / or a combination of hardware, software, and / or firmware in a large number of computing devices. In some example embodiments, the example computer-implemented method includes: receiving input speech; generating a recognition hypothesis based on the input speech and using a recognition algorithm, wherein the recognition hypothesis includes at least one pairing, the at least one pairing including a word and an initial confidence score associated with the word; determining whether the recognition hypothesis matches the expected response based at least in part on comparing the word with a corresponding word in the expected response; and in response to determining that the recognition hypothesis matches the expected response, updating statistics associated with the word, wherein the updated statistics are configured to update a confidence score adjustment value associated with the word.

[0005] In some example embodiments, the statistics associated with the terms include one or more of a sum metric, a count metric, or a sum of squares metric.

[0006] In some example embodiments, statistics for a word are determined based at least in part on each confidence score associated with the word in a recognition hypothesis and / or each initial confidence score associated with the word in at least one historical recognition hypothesis.

[0007] In some example embodiments, the example computer-implemented method further comprises storing the updated statistical data in a data repository.

[0008] In some example embodiments, the example computer-implemented method further includes: generating an adjusted confidence score for the term by applying the confidence score adjustment value for the term to an initial confidence score associated with the term; and determining whether the adjusted confidence score satisfies a rejection threshold for the term.

[0009] In some example embodiments, the recognition hypothesis is rejected in response to determining that at least one term in the recognition hypothesis is associated with an adjusted confidence score that fails to meet a rejection threshold for the at least one term.

[0010] In some example embodiments, the recognition hypothesis is accepted in response to determining that each term in the recognition hypothesis is associated with an adjusted confidence score that satisfies a corresponding rejection threshold for each term.

[0011] In some example embodiments, the example computer-implemented method further includes: generating an updated confidence score adjustment value for the word based at least in part on the updated statistical data; and applying the updated confidence score adjustment value to a subsequent recognition hypothesis that includes another instance of the same word and an associated initial confidence score by adjusting the initial confidence score for the word in the subsequent recognition hypothesis based at least in part on the updated confidence score adjustment value.

[0012] In some example embodiments, the updated confidence score adjustment value for a word is generated based at least in part on a condition indicating that an average adjusted confidence score for the word based on a set of historical recognition hypotheses satisfies a rejection threshold for the word at a predefined value.

[0013] In some example embodiments, the updated confidence score adjustment value for a word is generated based at least in part on a condition indicating that the average adjusted confidence score for the word based on the recognition hypothesis and the historical recognition hypothesis corresponds to N standard deviations above a rejection threshold.

[0014] In some example embodiments, in response to determining that a term is associated with an adjusted confidence score that satisfies a rejection threshold for the term, the term is accepted.

[0015] According to another aspect of the present disclosure, a device for adapting a speech recognition confidence score is provided. In some embodiments, the device includes at least one processor and at least one non-transitory memory having computer-coded instructions stored thereon. The computer-coded instructions, when executed by at least one processor, cause the device to perform any of the example computer-implemented methods described herein. In some other embodiments, the device includes means for performing each step of any of the computer-implemented methods described herein.

[0016] According to another aspect of the present disclosure, a computer program product for adapting a speech recognition confidence score is provided. In some embodiments, the computer program product includes at least one non-transitory computer-readable storage medium having computer program code stored thereon. The computer program code, when executed by at least one processor, is configured to perform any of the example computer-implemented methods described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Having thus generally described embodiments of the present disclosure, reference will now be made to the accompanying drawings, which are not necessarily drawn to scale, and in which:

[0018] Figure 1 shows a block diagram of a system that may be specially configured, and within which embodiments of the present disclosure may operate;

[0019] Figure 2 A block diagram illustrating an example apparatus embodying a speech recognition system according to at least one embodiment of the present disclosure is shown;

[0020] Figure 3 shows a visualization of an example data environment for adapting speech recognition confidence scores in accordance with at least one embodiment of the present disclosure;

[0021] Figure 4 A flowchart depicting example operations of an example process for adjusting confidence scores in accordance with at least one embodiment of the present disclosure is shown; and

[0022] Figure 5 A flow chart depicting example operations of an example process for adapting speech recognition confidence scores to a user in accordance with at least one embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0023] Embodiments of the present disclosure will now be described more fully below with reference to the accompanying drawings, in which some (but not all) embodiments of the present disclosure are shown. In fact, the embodiments of the present disclosure can be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein; on the contrary, these embodiments are provided so that the present disclosure will satisfy applicable legal requirements. The same mark always refers to the same element. Unless otherwise indicated, the term "or" used in this article is both an alternative meaning and a combined meaning. The terms "illustrative" and "example" are examples for which there is no indication of a quality level. Terms such as "calculate", "determine", "generate" and / or similar words are used interchangeably in this article to refer to the creation, modification or identification of data. Further, "based on", "partially based on", "at least based on" and / or similar words are used interchangeably in this article in an open manner so that they do not indicate that they are based only on or only on one or more elements referenced, unless so indicated. The same mark always refers to the same element.

[0024] Overview and technical improvements

[0025] Speech recognition is relied upon in a variety of environments for a variety of reasons. For example, speech recognition has simplified many tasks in the workplace by allowing hands-free communication with computers. Workers can input data by voice using a speech recognizer, and commands or instructions can be communicated to the worker via a speech synthesizer. Speech recognition is particularly well-suited for mobile computing devices. For example, wireless wearable terminals can provide workers performing work-related tasks with the desired computing and data processing capabilities while providing them with enhanced mobility within the workplace. One particular area in which workers rely heavily on such wireless wearable terminals is inventory management. Generally speaking, inventory-driven industries rely on computerized inventory management systems for performing a variety of different tasks, such as food and retail product distribution, manufacturing, and quality control.

[0026] In one scenario, for example, an overall integrated management system may involve a combination of a central computer system for tracking and management, and personnel in the form of order fillers, pickers, and other workers who use and interface with the computer system. The workers handle the manual aspects of the integrated management system under the command and control of information transmitted from the central computer system to wireless wearable terminals.

[0027] For example, when workers complete their assigned tasks, a two-way communication flow of information is exchanged between the wireless wearable terminal and the central computer system via a wireless network. The information received by each wireless wearable terminal from the central computer system is converted into voice instructions or text commands for the corresponding workers. Typically, workers wear a headset coupled to a wearable device, which has a microphone for voice data input and a receiver speaker for audio output feedback. The response from the worker is input into the wireless wearable terminal by the headset microphone and transmitted from the wireless wearable terminal to the central computer system. Through the headset microphone, workers can ask questions, report the progress of completing their assigned tasks, and report working conditions (such as inventory shortages). Using this type of wireless wearable terminal, workers can perform their assigned tasks almost without using their hands, without having to deal with equipment or paperwork to carry with them. Because manual data entry is eliminated or at least reduced, workers can perform their tasks faster, more accurately, and more efficiently.

[0028] An illustrative example of a worker task set suitable for a wireless wearable terminal with voice capability can involve initially welcoming a worker into a computerized inventory management system and defining a specific task or order, such as loading a specific truck scheduled to depart from a warehouse. The worker can then answer the specific area (e.g., freezer) in which they will work for that order. The system then voice-guides the worker to a specific aisle and bin to pick a specific amount of items. The worker then voice-confirms the location and quantity of the items picked. The system can then guide the worker to the loading dock or loading compartment for a specific truck to receive the order. As can be appreciated, the specific communications exchanged between the wireless wearable terminal and the central computer system can be task-specific and highly variable. During the interaction between the worker and the system, there may be instances where the system expects a specific response from the worker (e.g., when the worker is confirming his or her location).

[0029] While existing speech recognition systems generally address the task of determining spoken input and recognizing speech, there are still some shortcomings in such systems, such as errors in false rejection, false acceptance, and out-of-vocabulary insertion, to name a few. In this regard, speech is not always interpreted or recognized correctly, especially when different users with different speech patterns interact with the system. The embodiments of the present disclosure address the above-mentioned shortcomings and other challenges and difficulties associated with speech recognition. Specifically, the embodiments of the present disclosure utilize expected responses to improve the accuracy of speech recognition systems.

[0030] Embodiments of the present disclosure provide for adapting speech recognition confidence scores to specific users or user sets using expected responses. In some embodiments, a recognition hypothesis is generated for the received input speech using a recognition algorithm. The recognition hypothesis may include one or more words and a confidence score associated with each of the one or more words. For example, each of the one or more words generated is assigned or otherwise associated with a confidence score that quantitatively indicates the likelihood (e.g., chance) that a particular word is accurately recognized. In some embodiments, for each word in the hypothesis, a corresponding confidence score adjustment value is determined (e.g., calculated) and / or tracked, and applied to the word to generate an adjusted confidence score based on an initial confidence score. The adjusted confidence score associated with the word is then compared with a rejection threshold. If the adjusted confidence score meets (e.g., exceeds, is lower than, is equal to, etc.) the rejection threshold, the corresponding word is accepted. If the adjusted confidence score fails to meet the rejection threshold, the specific word is rejected. In some example embodiments, in response to determining that at least one word in the recognition hypothesis is associated with an adjusted confidence score that fails to meet the rejection threshold, the recognition hypothesis is rejected. In some example embodiments, the recognition hypothesis is accepted in response to determining that each word in the recognition hypothesis is associated with an adjusted confidence score that satisfies a rejection threshold. In some examples, the rejection threshold can be different for different words.

[0031] In order to improve accuracy, embodiments of the present disclosure utilize expected responses to update confidence score adjustment values ​​for words in a manner that reduces the risk of false rejection, the risk of false acceptance, the risk of inserting words beyond vocabulary, and / or similar. In some embodiments, the recognition hypothesis is compared with the expected response to determine whether each of the one or more words in the hypothesis matches the corresponding word in the expected response. In some embodiments, for each word, in response to determining that the word matches the corresponding word in the expected response, the confidence score adjustment value for the word and / or the statistics associated with the word are updated. Alternatively or additionally, in some embodiments, the recognition hypothesis is compared with the expected response to determine a complete match (e.g., determining that all words in the expected response match the recognition hypothesis), and in response to determining that the recognition hypothesis matches the expected response, the confidence score adjustment value for each word and / or the statistics associated with each word are updated. The updated confidence score adjustment value can be applied to subsequent occurrences of the word in subsequent recognition hypotheses, which include another instance of the same word. In some embodiments, for example, updated confidence score adjustments for a term are calculated in real time or near real time relative to subsequent recognition hypotheses using updated statistics associated with the term.

[0032] By updating the confidence score adjustment value based on the expected response, the embodiments of the present disclosure reduce the risk of confidence score adaptation being far from adaptation. For example, based on the use of expected responses and the requirement that the expected responses match the recognition hypothesis, the embodiments prevent most incorrect and / or invalid data from being used in the statistics (e.g., statistical data) used for adaptation. For example, if background noise causes the word "six" to be recognized as having a poor score, in the absence of an expected response check, the recognition system will adapt to the expected word "six" having a poor score and will allow more "sixes" with lower scores to be recognized, thereby making "six" more likely to be incorrectly recognized in response to background noise.

[0033] Thus, in an embodiment, adapting the confidence score using the expected response allows the adaptation to occur with a reduced risk of the confidence score adaptation being off, thereby improving recognition accuracy and eliminating the need for the user to repeat the utterance for speech recognition processing by the speech recognition system. By eliminating the need for the user to repeat the utterance for speech recognition processing, embodiments of the present disclosure improve worker (e.g., user) productivity and computing efficiency, and reduce network traffic.

[0034] definition

[0035] As used herein, the term "comprising" means including but not limited to, and should be interpreted in the manner in which it is typically used in a patent context. The use of broader terms such as "including," "comprising," and "having" should be understood to provide support for narrower terms such as "consisting of," "consisting essentially of," and "composed essentially of."

[0036] The phrases "in one embodiment," "according to one embodiment," "in some embodiments," etc. generally mean that the particular feature, structure, or characteristic following the phrase can be included in at least one embodiment of the present disclosure, and may be included in more than one embodiment of the present disclosure (importantly, such phrases are not necessarily referring to the same embodiment).

[0037] The word “example” or “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any implementation described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other implementations.

[0038] If the specification states that a component or feature "may," "could," "might," "should," "will," "preferably," "likely," "typically," "optionally," "for example," "usually," or "might" (or other such language) be included or have a characteristic, that particular component or feature is not required to be included or have that characteristic. Such a component or feature may optionally be included in some embodiments, or it may be excluded.

[0039] The term "recognition hypothesis" refers to the output of a recognizer system associated with a speech input to the recognition system. In some embodiments, a recognition hypothesis includes a sequence of one or more words and an associated confidence score for each of the one or more words.

[0040] The term "expected response" refers to a sequence of one or more words that a user is most likely to speak at a particular execution point in a workflow. In some embodiments, the expected response is used to determine whether to update the confidence score adjustment value associated with the word.

[0041] The term "initial confidence score" refers to electronically managed data that indicates the likelihood (e.g., chance) that a word hypothesized by a recognizer system is accurately recognized. For example, in some embodiments, the initial confidence score corresponds to the original confidence score associated with the word hypothesized by the recognizer system.

[0042] The term "confidence score adjustment value" refers to electronically managed data representing a value applied to an initial confidence score associated with a term to adapt the confidence score for the term to one or more users. In some embodiments, confidence score adjustment values ​​are maintained for different instances across recognition hypotheses derived from spoken audio data.

[0043] The term "updated confidence score adjustment value" refers to electronically managed data representing a confidence score adjustment value that is modified based at least in part on statistical data associated with the corresponding term.

[0044] The term "model" refers to one or more algorithms, statistics, and / or machine learning models configured to generate an output. In some embodiments, one or more models are trained to generate updated confidence score adjustment values.

[0045] Example systems and apparatus of the present disclosure

[0046] Figure 1 A block diagram of a system that may be specifically configured, within which embodiments of the present disclosure may operate, is shown. Specifically, Figure 1An example system 100 is depicted. As shown, the system 100 includes a speech recognition system 102 configured to communicate with one or more client systems 104. In some embodiments, the speech recognition system 102 and the client systems 104 communicate via one or more communication networks (e.g., communication network 106).

[0047] It should be understood that in some embodiments, the communication network 106 is embodied in any of a large number of network configurations. In some embodiments, the communication network 106 embodies a public network (e.g., the Internet). In some embodiments, the communication network 106 embodies a private network (e.g., an internal localized or closed network between specific devices). In some other embodiments, the communication network 106 embodies a hybrid network (e.g., a network that enables internal communication between devices that are specifically connected and external communication with other devices). In some embodiments, the communication network 106 includes one or more base stations, repeaters, routers, switches, cell towers, communication cables and / or associated routing stations, etc. In some embodiments, the communication network 106 includes one or more user-controlled computing devices (e.g., a router and / or modem owned by the user) and / or one or more external utility devices (e.g., an Internet service provider communication tower and / or other equipment).

[0048] Each of the components of the system 100 is communicatively coupled to transmit and / or receive data from one another via the same or different wireless and / or wired networks embodying the communication network 106. Such configurations include, but are not limited to, wired or wireless personal area networks (PANs), local area networks (LANs), metropolitan area networks (MANs), wide area networks (WANs), and the like. Figure 1 Certain system entities are shown as separate, independent entities communicating via communication network 106, but various embodiments are not limited to this architecture. In other embodiments, one or more computing entities share one or more components, hardware, etc., or are otherwise embodied by a single computing device, such that connections between computing entities via communication network 106 are altered and / or unnecessary.

[0049] In some embodiments, the client system is embodied in an indoor system. In one embodiment, the client system is embodied in an indoor system within or associated with a specific industrial system, for example, including any number of third-party devices, sensors, etc. Additionally or alternatively, in some embodiments, the client system 104 and the speech recognition system 102 are embodied as an indoor system associated with a specific system, which is associated with a warehouse, a factory, etc. In some embodiments, the indoor system is communicatively coupled via at least one wired device. Additionally or alternatively, one or more of such systems can be remote from each other, for example, where the client system represents an indoor system corresponding to, for example, a specific industrial system, and where the speech recognition system 102 is a cloud system or is otherwise remotely located from the client system.

[0050] In some embodiments, client system 104 includes any number of computing devices, systems, physical components, etc. that facilitate the generation and / or use of speech signals corresponding to utterances by a user. For example, in some embodiments, client system 104 embodies an input speech device configured to capture speech signals from a user. In some embodiments, client system 104 includes one or more end-user terminals, servers, etc., configured to perform such functions.

[0051] The speech recognition system 102 includes one or more computing devices, systems, etc. embodied in hardware, software, firmware, and / or a combination thereof, which perform speech recognition based on a speech signal. Additionally or alternatively, in some embodiments, the speech recognition system 102 includes one or more computing devices, systems, etc. embodied in hardware, software, firmware, and / or a combination thereof, which perform the generation and / or maintenance of confidence score adjustment values ​​for one or more words, and / or the generation and / or maintenance of statistical data associated with one or more words. In some embodiments, the speech recognition system 102 includes one or more specially configured application servers, database servers, end-user devices, cloud computing systems, etc. Additionally or alternatively, in some embodiments, the speech recognition system 102 includes one or more user devices that enable implementation of access to functionality provided by the speech recognition system 102 (e.g., via web applications, local applications, etc.). Still alternatively or additionally, in some embodiments, a client system is specially configured to provide access to the functionality of the client system.

[0052] In some embodiments, the speech recognition system 102 in conjunction with the client system is configured to promote speech recognition and / or adapt the confidence score of the recognizer system (associated with the speech recognizer system) to the user. In some embodiments, the speech recognition system 102 is configured to promote the generation, storage and / or maintenance of recognition hypotheses for input speech. In some embodiments, the speech recognition system 102 is configured to update (e.g., modify) the confidence score adjustment value associated with the word. Alternatively or additionally, in some embodiments, the speech recognition system 102 is configured to update the statistical data associated with the word.

[0053] In some embodiments, the speech recognition system 102 and the client systems 104 communicate with each other to perform the various actions described herein. For example, in some embodiments, the speech recognition system 102 and one or more of the client systems 104 communicate to generate and store recognition hypotheses. In some embodiments, the speech recognition system 102 and one or more of the client systems 104 communicate to display, configure, and / or update confidence score adjustment values ​​utilized by the speech recognition system to determine whether to accept or reject words hypothesized by a recognizer system associated with the speech recognition system. In some embodiments, the speech recognition system 102 and one or more of the client systems 104 communicate to display, configure, and / or update statistics utilized by the speech recognition system to calculate confidence score adjustment values.

[0054] Figure 2 A block diagram of an example device that may be specifically configured in accordance with at least one example embodiment of the present disclosure is shown. Specifically, Figure 2 An example speech recognition device ("device 200") is depicted that is specifically configured according to at least one example embodiment of the present disclosure. In some embodiments, the speech recognition system 102 and / or portions thereof are comprised of one or more systems (such as Figure 2 The apparatus 200 is embodied in the apparatus 200 depicted and described herein. The apparatus 200 includes a processor 202, a memory 204, an input / output circuit 206, a communication circuit 208, a hypothesis generation circuit 210, a hypothesis acceptance circuit 212, a hypothesis matching circuit 214, and / or an adaptation circuit 216. In some embodiments, the apparatus 200 is configured to use one or more of the sets of circuits embodied by the processor 202, the memory 204, the input / output circuit 206, the communication circuit 208, the hypothesis generation circuit 210, the hypothesis acceptance circuit 212, the hypothesis matching circuit 214, and / or the adaptation circuit 216 to perform and implement the operations described herein.

[0055] In general, the term computing entity (or reference to an "entity" other than a user), device, system and / or similar terms used interchangeably herein can refer to, for example, one or more computers, computing entities, desktop computers, mobile phones, tablet computers, tablet phones, notebook computers, laptop computers, distributed systems, items / equipment, terminals, servers or server networks, blade servers, gateways, switches, processing equipment, processing entities, set-top boxes, repeaters, routers, network access points, base stations, etc. and / or any combination of devices or entities suitable for performing the functions, operations and / or processes described herein. Such functions, operations and / or processes can include, for example, transmitting, receiving, operating, processing, displaying, storing, determining, creating / generating, monitoring, evaluating, comparing and / or similar terms used interchangeably herein. In one embodiment, these functions, operations and / or processes can be performed on data, content, information and / or similar terms used interchangeably herein. In this regard, as described herein, device 200 embodies a specific, specially configured computing entity that is converted into a computing entity that performs the specific operations described herein and provides specific advantages associated therewith.

[0056] While these components are described with respect to functional limitations, it should be understood that a particular implementation necessarily includes the use of specific computing hardware. It should also be understood that, in some embodiments, certain components described herein include similar or common hardware. For example, in some embodiments, both sets of circuits utilize the use of the same processor, network interface, storage medium, etc. to perform their associated functions, such that duplicate hardware is not required for each set of circuits. Thus, it should be understood that use of the term "circuitry" as used herein with respect to components of the apparatus described herein includes specific hardware configured to perform the functions associated with the specific circuits as described herein.

[0057] In particular, the term "circuitry" should be broadly interpreted to include hardware, and in some embodiments, software for configuring hardware. For example, in some embodiments, "circuitry" includes processing circuitry, storage media, network interfaces, input / output devices, etc. Alternatively or additionally, in some embodiments, other elements of apparatus 200 provide or supplement the functionality of another particular set of circuits. For example, in some embodiments, processor 202 provides processing functionality to any of a set of circuits, memory 204 provides storage functionality to any of a set of circuits, communication circuitry 208 provides network interface functionality to any of a set of circuits, and / or the like.

[0058] In some embodiments, the processor 202 (and / or a coprocessor, or any other processing circuitry assisting the processor or otherwise associated with the processor) communicates with the memory 204 via a bus for transferring information between components of the apparatus 200. In some embodiments, for example, the memory 204 is non-transitory and may include, for example, one or more volatile and / or non-volatile memories. In other words, for example, in some embodiments, the memory 204 includes or embodies an electronic storage device (e.g., a computer-readable storage medium). In some embodiments, the memory 204 is configured to store information, data, content, applications, instructions, or the like for enabling the apparatus 200 to perform various functions according to example embodiments of the present disclosure.

[0059] The processor 202 can be embodied in a variety of different ways. For example, in some example embodiments, the processor 202 includes one or more processing devices that are configured to execute independently. Additionally or alternatively, in some embodiments, the processor 202 includes one or more processors that are configured in series via a bus to enable independent execution of instructions, pipelining, and / or multithreading. The use of the terms "processor" and "processing circuitry" should be understood to include a single-core processor, a multi-core processor, multiple processors within the device 200, and / or one or more remote processors or "cloud" processors external to the device 200.

[0060] In an example embodiment, the processor 202 is configured to execute instructions stored in the memory 204 or otherwise accessible to the processor. Alternatively or additionally, in some embodiments, the processor 202 is configured to execute hard-coded functionality. Thus, whether configured by hardware methods or software methods, or by a combination thereof, the processor 202 represents an entity (e.g., physically embodied in circuit form) capable of performing operations in accordance with embodiments of the present disclosure while being configured accordingly. Alternatively or additionally, as another example, in some example embodiments, when the processor 202 is embodied as an executor of software instructions, these instructions specifically configure the processor 202 to perform algorithms embodied in the specific operations described herein when executing such instructions.

[0061] As a specific example embodiment, processor 202 is configured to perform and perform speech recognition and the various operations associated with speech recognition confidence score being adapted to the user.In some such embodiments, processor 202 includes hardware, software, firmware and / or a combination thereof that generates the recognition hypothesis for input speech.In some such embodiments, the recognition hypothesis includes the sequence of one or more words and the associated confidence score for each word in one or more words. Additionally or alternatively, in some such embodiments, processor 202 includes hardware, software, firmware and / or a combination thereof, which adjusts the confidence score associated with the word at least in part based on the confidence score adjustment value associated with the word of hypothesis. Additionally or alternatively, in some such embodiments, processor 202 includes hardware, software, firmware and / or a combination thereof, which will compare with a rejection threshold through adjusted confidence score, to determine whether to accept or reject the word. Additionally or alternatively, in some such embodiments, the processor 202 includes hardware, software, firmware, and / or a combination thereof that compares a recognition hypothesis to an expected response match to determine whether a word of the recognition hypothesis matches a corresponding word in the expected response. In some such embodiments, the processor 202 includes hardware, software, firmware, and / or a combination thereof that generates an updated confidence score adjustment value for a word based at least in part on statistics associated with the hypothesized word (e.g., using one or more models).

[0062] In some embodiments, the apparatus 200 includes a communication circuit 208. The communication circuit 208 includes any means, such as a device or circuit embodied in hardware or a combination of hardware and software, that is configured to receive and / or transmit data from / to a network and / or any other device, circuit, or module that communicates with the apparatus 200. In this regard, in some embodiments, the communication circuit 208 includes, for example, a network interface for enabling communication with a wired or wireless communication network. Additionally or alternatively, in some embodiments, the communication circuit 208 includes one or more network interface cards, antennas, buses, switches, routers, modems, and supporting hardware, firmware, and / or software, or any other device suitable for enabling communication via one or more communication networks. Additionally or alternatively, the communication circuit 208 includes circuitry for interacting with an antenna and / or other hardware or software to enable transmission of signals via the antenna or processing of signals received via the antenna. In some embodiments, the communication circuit 208 enables transmission of data to a user device, one or more assets or accompanying sensors, and / or other external computing devices in communication with the apparatus 200, and / or reception of data from a user device, one or more assets or accompanying sensors, and / or other external computing devices in communication with the apparatus 200.

[0063] In some embodiments, the device 200 includes a hypothesis generation circuit 210. The hypothesis generation circuit 210 includes hardware, software, firmware, and / or a combination thereof that supports the generation and use of recognition hypotheses for input speech. For example, in some embodiments, the hypothesis generation circuit 210 includes hardware, software, firmware, and / or a combination thereof that receives input speech from, for example, the client system 104. Additionally or alternatively, in some embodiments, the hypothesis generation circuit 210 includes hardware, software, firmware, and / or a combination thereof that processes the input speech to generate a recognition hypothesis comprising a sequence of one or more words and an associated confidence score for each of the one or more words. In some embodiments, the hypothesis generation circuit 210 includes a separate processor, a specially configured field programmable gate array (FPGA), or a specially programmed application-specific integrated circuit (ASIC).

[0064] In some embodiments, the device 200 includes a hypothesis acceptance circuit 212. The hypothesis acceptance circuit 212 includes hardware, software, firmware, and / or a combination thereof that supports determining whether a recognition hypothesis should be accepted or rejected based at least in part on an adjusted confidence score associated with one or more words in the recognition hypothesis. For example, in some embodiments, the hypothesis acceptance circuit 212 includes hardware, software, firmware, and / or a combination thereof that adjusts an initial confidence score associated with a word based at least in part on a confidence score adjustment value associated with the word of the hypothesis. In some embodiments, the hypothesis acceptance circuit 212 includes hardware, software, firmware, and / or a combination thereof that compares the adjusted confidence score associated with each word in the recognition hypothesis with a rejection threshold to determine whether to accept the recognition hypothesis. In some embodiments, the hypothesis acceptance circuit 212 includes a separate processor, a specially configured field programmable gate array (FPGA), or a specially programmed application specific integrated circuit (ASIC).

[0065] In some embodiments, the apparatus 200 includes a hypothesis matching circuit 214. The hypothesis matching circuit 214 includes hardware, software, firmware, and / or a combination thereof that supports determining whether a word in a recognition hypothesis matches a corresponding word in an expected response. For example, in some embodiments, the hypothesis matching circuit 214 includes hardware, software, firmware, and / or a combination thereof that maintains at least one data repository comprising any number of expected responses. Additionally or alternatively, in some embodiments, the hypothesis matching circuit 214 includes hardware, software, firmware, and / or a combination thereof that receives a recognition hypothesis, for example, from the hypothesis generation circuit 210 and compares at least one word in the recognition hypothesis with a corresponding word in the expected response. Additionally or alternatively, in some embodiments, the hypothesis matching circuit 214 includes hardware, software, firmware, and / or a combination thereof that retrieves a recognition hypothesis from a data repository, such as a data repository maintained by the hypothesis generation circuit 210. In some embodiments, the hypothesis matching circuit 214 includes a separate processor, a specially configured field programmable gate array (FPGA), or a specially programmed application-specific integrated circuit (ASIC).

[0066] In some embodiments, the device 200 includes an adaptation circuit 216. The adaptation circuit 216 includes hardware, software, firmware and / or a combination thereof that supports confidence score adjustment values ​​such as storage, maintenance, and updating. In some embodiments, the adaptation circuit 216 includes hardware, software, firmware and / or a combination thereof that generates, updates and / or stores statistics associated with each word in the recognition hypothesis. In some embodiments, the adaptation circuit 216 includes hardware, software, firmware and / or a combination thereof that updates the confidence score adjustment value associated with the word to generate an updated confidence score adjustment value for the word. For example, in some embodiments, the adaptation circuit 216 applies statistics associated with the word in the recognition hypothesis to historical statistics associated with the word to generate an updated confidence score adjustment value for the word. In some embodiments, the hypothesis matching circuit 214 includes a separate processor, a specially configured field programmable gate array (FPGA), or a specially programmed application specific integrated circuit (ASIC).

[0067] Additionally or alternatively, in some embodiments, two or more of the sets of circuits embodying the processor 202, memory 204, input / output circuitry 206, communication circuitry 208, hypothesis generation circuitry 210, hypothesis acceptance circuitry 212, hypothesis matching circuitry 214, and / or adaptation circuitry 216 are combinable. Alternatively or additionally, in some embodiments, one or more of the sets of circuits perform some or all of the described functionality associated with another component. For example, in some embodiments, two or more of the sets of circuits embodied by the processor 202, memory 204, input / output circuitry 206, communication circuitry 208, hypothesis generation circuitry 210, hypothesis acceptance circuitry 212, hypothesis matching circuitry 214, and / or adaptation circuitry 216 are combined into a single module embodied in hardware, software, firmware, and / or a combination thereof. Similarly, in some embodiments, one or more of the circuit sets (e.g., hypothesis generation circuitry 210, hypothesis acceptance circuitry 212, hypothesis matching circuitry 214, and / or adaptation circuitry 216) are combined with processor 202 such that processor 202 performs one or more of the operations described above with respect to each of these circuit sets embodied by hypothesis generation circuitry 210, hypothesis acceptance circuitry 212, hypothesis matching circuitry 214, and / or adaptation circuitry 216.

[0068] Example Data Environment and Architecture for This Disclosure

[0069] Having described example systems and devices of the present disclosure, example data architectures, data environments, and data flows will now be described. In some embodiments, a data architecture represents data objects maintained and processed in a particular computing environment. In some embodiments, the computing environment is maintained via hardware, software, firmware, and / or a combination thereof that execute one or more software applications that manage such data. For example, in some embodiments, the device 200 executes one or more software applications that maintain a data architecture as depicted and described to perform, individually or in combination with each other, functionality as depicted and described with respect to adapting a speech recognition confidence score to a user.

[0070] Figure 3A visualization of an example data environment for adapting a speech recognition confidence score to a user in accordance with at least one embodiment of the present disclosure is shown. Specifically, the data environment for adapting the speech recognition confidence score to the user is performed by the speech recognition system 102. In some embodiments, the speech recognition system 102 is embodied by the device 200, as depicted and described herein. In some embodiments, the speech recognition system 102 generates a presentation of or otherwise provides access to one or more user interfaces that are specifically configured to enable input of data.

[0071] As shown in the figure, the recognizer module 304 of speech recognition system 102 receives the input speech 302 corresponding to one or more words spoken by the user, and processes the input speech 302 to generate recognition hypothesis 306. In some embodiments, recognition hypothesis includes a sequence of one or more words that are assumed by the speech recognition system to be the words spoken by the user. Additionally or alternatively, in some embodiments, recognition hypothesis includes a confidence score for each word in the one or more words in the recognition hypothesis. As shown in the figure, in some embodiments, the confidence score associated with each word in the recognition hypothesis corresponds to or otherwise represents an initial confidence score for the corresponding word. For example, in some embodiments, recognition hypothesis 306 includes one or more words and an initial confidence score 328 assigned to each word in the one or more words by the recognizer module 304.

[0072] In some embodiments, the recognizer module 304 uses a recognition algorithm to generate recognition hypotheses 306. For example, the recognition algorithm can rely on one or more probability models and / or other modeling techniques to generate recognition hypotheses 306. A non-limiting example of a modeling technique that can be utilized by the recognizer module 304 includes a hidden Markov model (HMM) technology. In speech recognition, the HMM model uses a sequence of states to describe vocabulary items, which can be words, phrases, or sub-word units. As used herein, the term "word" is used to represent a vocabulary item, and therefore can mean a word, a segment or part of a word, or a synthetic word (such as "next bin" or "say it again"). Therefore, the term "word" is not limited to only a single word. Each state in the HMM can represent one or more acoustic events and is used to assign a probability to each observed feature vector. Therefore, the path through the HMM state can produce a series of probability indications of acoustic feature vectors. The model is searched so that different competing hypotheses (or paths) are scored (e.g., acoustic matching or acoustic search). State S can be reached at time T via multiple different paths. For each path that reaches a specific state at a specific time, a path probability is calculated. Using the Viterbi algorithm, a probability can be assigned to each path through the HMM. In particular, a probability can be assigned to the best path. In addition, a probability can be assigned to each word in the best path. Each of these probabilities can be used as a confidence score, or combined with other measurements, estimates, or numbers to derive a confidence score. The path with the highest confidence score (the best hypothesis) can then be further analyzed. It should be understood that other speech recognition models are also contemplated within the scope of the present invention; for example, template matching dynamic time warping (DTW) and neural networks are two such exemplary alternative modeling techniques.

[0073] As shown, input speech 302 is received from the client system 104. The input speech 302 can be captured in various ways by the client system 104. For example, a microphone or other electro-acoustic device senses speech from a user and transmits the input speech to the recognizer module 304.

[0074] In some embodiments, the recognizer module 304 includes one or more components, such as a signal processor, a feature generator, a decoder, etc., which are used by the recognizer module 304 to process the input speech and generate recognition hypotheses 306. In some embodiments, the signal processor includes an analog-to-digital converter, a filter, an equalizer circuit, and / or other components that convert the received input speech 302 into a digitized data stream, which can be divided into individual units for analysis, such as, for example, frames. In some such embodiments, the signal processor is configured to divide the generated digitized data stream into a sequence of frames, each of which is then processed by a feature generator to generate features for the frame. The generated features may, for example, include acoustic features of the frame. In some such embodiments, the features are represented in the form of vectors, matrices, or otherwise organized sets of numbers. The feature generator may utilize one or more of a variety of techniques to represent the speech. As an example, the feature generator may utilize linear predictive coding (LPC) coefficients to represent the speech. As another example, the feature generator may utilize Mel-frequency cepstral coefficients to represent the speech. In some such embodiments, the decoder is configured to generate recognition hypotheses 306 based at least in part on the features and using a recognition algorithm.

[0075] In some embodiments, for each word in the recognition hypothesis 306, the initial confidence score 328 associated with the corresponding word is adjusted at least in part based on the confidence score adjustment value 308 associated with the corresponding word. For example, as shown, in some embodiments, the confidence score adjustment module 309 of the speech recognition system 102 is configured to retrieve the confidence score adjustment value 308 associated with each word in the recognition hypothesis 306, for example, from the confidence score adjustment repository 310. In some embodiments, the confidence score adjustment module 309 is configured to calculate the confidence score adjustment value 308 in real time (e.g., on demand). In some such embodiments, the confidence score adjustment module 309 can be configured to retrieve the statistics associated with the word from the statistics repository 330 to calculate the confidence score adjustment value 308. In some embodiments, the recognition system 102 includes an update module 340 that is configured to update the statistics associated with the word, for example, in response to determining that the recognition hypothesis 306 matches the expected response 314. For example, the updating module 340 can be configured to generate updated statistical data 332 and store the updated statistical data 332 in the statistical data repository 330 in response to determining that the recognition hypothesis 306 matches the expected response 314. In some such embodiments, the speech recognition system 102 may or may not include the confidence score adjustment repository 310. In some embodiments, in embodiments that include the confidence score adjustment repository 310, the updating module 340 can be configured to generate an updated confidence score adjustment value 322 and store the updated confidence score adjustment value 322 in the confidence score adjustment repository 310. In some embodiments, the confidence score adjustment repository 310 can include a data repository.

[0076] In some embodiments, for each word in the recognition hypothesis 306, a confidence score adjustment value 308 is applied to the initial confidence score for the corresponding word to generate an adjusted confidence score 312. In some embodiments, the adjusted confidence score 312 for each word is compared with a rejection threshold 317 to determine whether to accept or reject the word. As shown, in some embodiments, the speech recognition system 102 utilizes an acceptance algorithm 316 to determine whether to accept or reject a particular word in the recognition hypothesis 306 based at least in part on a comparison of the particular word with the rejection threshold 317. In some embodiments, if the adjusted confidence score 312 for each word meets the rejection threshold 317, the recognition hypothesis 306 is accepted as recognized speech. In some embodiments, if the adjusted confidence score 312 for at least one word fails to meet the rejection threshold 317, the recognition hypothesis 306 is rejected.

[0077] As described herein, some embodiments of the present disclosure utilize expected responses to adapt speech recognition confidence scores to users, which in turn improves the accuracy of the speech recognition system. To provide context, in some examples, the most likely speech input to be received is known in advance, or can be determined and provided as an expected response. For example, when picking up parts from a container or placing items in a warehouse location, there may be a check digit that verifies the user's location or the operation to be performed by the user. During operation, the user may be prompted to go to a certain location and speak a check digit (or check phrase) displayed at the location and associated with the location. The speech recognition system attempts to recognize the response spoken by the user and compares it with the check digit or check phrase to confirm that the user is in the correct location before prompting the user to perform an operation (such as picking up a box of a product). As used herein, a "check digit" refers to a sequence of one or more words associated with a location, container, or slot for the purpose of verifying the user's location. For example, a "check digit" may be a three-digit number, a non-digital word, or a combination of these. Besides this exemplary environment, there are other scenarios where a particular response or series of responses is expected from a user.

[0078] In some embodiments, the expected response is retrieved from a data repository. For example, the speech recognition system 102 can be associated with a data repository that stores the expected response. Alternatively, the expected response can be dynamically determined before comparison.

[0079] In some embodiments, the speech recognition system 102 compares the recognition hypothesis 306 for the input speech 302 with the expected response 314 to determine a match. In some embodiments, comparing the recognition hypothesis 306 with the expected response includes, for example, comparing each word in the recognition hypothesis with a corresponding word in the expected response based at least in part on the position of the corresponding word in the sequence of one or more words in the recognition hypothesis. As shown, in some embodiments, the speech recognition system 102 utilizes a matching algorithm 318 to determine whether a word in the recognition hypothesis 306 matches a corresponding word in the expected response 314 and / or to determine whether the recognition hypothesis 306 matches the expected response 314.

[0080] In some embodiments, the matching algorithm 318 is configured to compare words (e.g., word tokens) to determine a match. In some embodiments, the matching algorithm 318 may include any conventional algorithm configured to compare a first group of word sequences with a second group of word sequences to determine a match relative to each pair of words compared and / or determine a match relative to a sequence pair of one or more words. In an example embodiment, the matching algorithm 318 is configured to compare words in a first group of word sequences (e.g., recognition hypotheses) with words in a second group of word sequences (e.g., expected responses) based on the position of the words in the corresponding set of word sequences. For example, in some embodiments, the first word in the first group of word sequences is compared with the first word in the second group of word sequences, the second word in the first group of word sequences is compared with the second word in the second group of word sequences, and the third word in the first group of word sequences is compared with the third word in the second group of word sequences. In some embodiments, the matching algorithm 318 is configured to generate an output indicating whether a particular word in the first group of word sequences matches a corresponding word in the second group of word sequences. Additionally or alternatively, in some embodiments, the matching algorithm 318 is configured to generate an output indicating whether the first set of word sequences matches the second set of word sequences based on the comparison of each corresponding word pair. In some embodiments, comparing the recognition hypothesis 306 with the expected response 314 using the matching algorithm 318 includes generating a match score for each pair of words compared and determining whether the match score meets a threshold. Additionally or alternatively, in some embodiments, comparing the recognition hypothesis 306 with the expected response using the matching algorithm 318 includes generating a match score for each pair of recognition hypothesis 306 and the expected response 314 and determining whether the match score meets a threshold. In some embodiments, if the match score meets the threshold, the recognition hypothesis 306 can be determined to match the expected response.

[0081] As described herein, in some embodiments, the speech recognition system 102 is configured to use the expected response and, at least in part, based on updating the confidence score adjustment value associated with the word, adapt the speech recognition confidence score to the user. For example, in some embodiments, if the word in the recognition hypothesis 306 is determined to match the corresponding word in the expected response 314, the speech recognition system 102 generates an updated confidence score adjustment value 322 for the word, wherein the updated confidence score adjustment value 322 can be configured to be applied to subsequent recognition hypotheses comprising another instance of the same word. In some embodiments, the speech recognition system 102 generates an updated confidence score adjustment value 322 for the word based at least in part on the statistical data associated with the word. Examples of statistical data include sum metrics, count metrics, square sum metrics, average metrics, etc. In some embodiments, the statistical data associated with a particular word include one or more of the sum metric, count metric, or square sum metric. It will be understood that in some embodiments, the statistical data associated with a given word can include other statistical metrics. In some embodiments, the speech recognition system 102 updates statistics associated with the words in the recognition hypothesis in response to determining that the recognition hypothesis fully matches the expected response (e.g., each word in the recognition hypothesis matches each corresponding word in the expected response, and / or other criteria). For example, in some embodiments, the speech recognition system 102 updates statistics associated with each word in the recognition hypothesis in response to determining that the recognition hypothesis fully matches the expected response. The updated statistics can be used to calculate an updated confidence score adjustment value 322.

[0082] In some embodiments, the statistics for a particular word are determined at least in part based on an initial set of confidence scores associated with the word in the recognition hypothesis 306 and / or an initial set of confidence scores associated with the word in historical recognition hypotheses (e.g., at least one historical recognition hypothesis). For example, in some embodiments, to update the statistics for a particular word, the speech recognition system 102 may apply the initial confidence score associated with the word to the historical statistics for the word to generate updated statistics. The speech recognition system 102 may generate an updated confidence score adjustment value 322 based at least in part on the updated statistics, wherein the updated confidence score adjustment value 322 is configured to be applied to a subsequent recognition hypothesis that includes another instance of the same word. In some embodiments, the speech recognition system 102 utilizes one or more models to generate updated statistics and / or updated confidence score adjustment values.

[0083] In some embodiments, the updated confidence score adjustment value 322 is generated at least in part based on one or more conditions. For example, one or more models are trained, configured, etc., to generate an updated confidence score adjustment value that meets one or more conditions. The one or more conditions can be configured to reduce the risk of false rejection, false acceptance, errors that exceed vocabulary insertion, and / or other factors that affect the accuracy of the speech recognition system. An example of such a condition represents that the average adjusted confidence score associated with the word is in a predefined value higher than a rejection threshold. In this regard, one or more models can be trained, configured, etc., to generate an updated confidence score adjustment value for a given word so that the average adjusted confidence score for the word based on the initial confidence score associated with the word and the historical confidence score meets the rejection threshold with a predefined value. Another example of such a condition represents that the average adjusted confidence score associated with the word is N standard deviations higher than the rejection threshold, where N can be configurable and can be any number (e.g., 1, 2, 2.5, 3, etc.). In this regard, one or more models can be trained, configured, etc. to generate updated confidence score adjustment values ​​for a given word such that the average adjusted confidence score for the word based on the initial confidence score and historical confidence scores associated with the word is N standard deviations above the rejection threshold.

[0084] In some embodiments, the confidence score adjustment repository 310 is updated to include updated confidence score adjustment values ​​322 for each word. Alternatively or additionally, in some embodiments, statistics associated with the words can be stored in a statistics repository 330. In such some embodiments, the speech recognition system 102 may or may not include the confidence score adjustment repository 310. For example, the confidence score adjustment values ​​can be determined (e.g., calculated) in real time (e.g., on demand), for example, based on statistics associated with the words that can be retrieved from the statistics repository 330.

[0085] Example procedures of the present disclosure

[0086] Having described example systems and apparatuses and data visualization according to the present disclosure, example processes of the present disclosure will now be discussed. It should be understood that each of the flowcharts depicts an example computer-implemented process that can be performed by one or more of the apparatuses, systems, devices, and / or computer program products described herein, for example, using one or more of their specifically configured components.

[0087] Although the example process depicts a particular sequence of operations, this sequence may be changed without departing from the scope of the present disclosure. For example, some of the depicted operations may be performed in parallel or in a different sequence that does not materially affect the functionality of the process.

[0088] Each frame indicates the operation of each process.This type of operation can be performed in any way in a variety of ways, including but not limited to performing in the order and manner described and depicted herein. In some embodiments, one or more frames in any process described herein occur between one or more frames of another process, occur before one or more frames of another process, occur in parallel with one or more frames of another process, and / or as a subprocess of a second process. Additionally or alternatively, in various embodiments, any process includes some or all of the operational steps described and / or depicted, includes one or more optional frames in some embodiments. About the flow chart shown herein, in some embodiments, one or more frames in the frame depicted are optional in some or all embodiments of the present disclosure. Optional frame is described with dotted line (or " dashed-dotted line "). Similarly, it should be understood that one or more operations in the operation of each flow chart can be combinable, replaceable and / or otherwise changed, as described herein.

[0089] Figure 4 A flowchart depicting example operations of an example process for adjusting confidence scores according to at least one embodiment of the present disclosure is shown. Specifically, Figure 4 An example process 400 is depicted. In some embodiments, process 400 is embodied by computer program code stored on a non-transitory computer-readable storage medium of a computer program product, the computer program code being configured to execute to implement the process as depicted and described. Alternatively or additionally, in some embodiments, process 400 is performed by one or more specially configured computing devices (such as apparatus 200, either alone or in communication with one or more other components, devices, systems, etc.). In this regard, in some such embodiments, apparatus 200 is specifically configured by computer-coded instructions (e.g., computer program instructions) stored thereon (e.g., stored in memory 204 and / or another component depicted and / or described herein and / or otherwise accessible to apparatus 200) to perform the operations depicted and described. In some embodiments, apparatus 200 communicates with one or more external devices, systems, devices, etc. to perform one or more of the operations as depicted and described. For example, in some embodiments, apparatus 200 communicates with at least one external data repository, client system, etc. to perform one or more of the operations as depicted and described. For purposes of simplicity of description, process 400 is described as being performed by apparatus 200 and from the perspective of the apparatus.

[0090] Although example process 400 depicts a particular sequence of operations, this sequence may be varied without departing from the scope of this disclosure. For example, some of the depicted operations may be performed in parallel or in a different sequence without substantially affecting the functionality of process 400. In other examples, different components of an example device or system implementing process 400 may perform functions substantially simultaneously or in a specific sequence.

[0091] According to some examples, the method includes receiving input speech at a speech recognition system at block 402. In some embodiments, the device 200 embodying the speech recognition system receives the input speech from a client system in response to a user's utterance. For example, the input speech can be received via a microphone associated with the client system.

[0092] According to some examples, the method includes: at box 404, generating a recognition hypothesis. In some embodiments, the device 200 generates a recognition hypothesis for the received input speech based on the received input speech and using a recognition algorithm. In some embodiments, for example, the device 200 uses a recognition algorithm to process one or more features extracted from the input speech to generate a recognition hypothesis. In some embodiments, the device 200 is configured to generate features for the input speech, wherein the feature can be represented as a vector (e.g., a feature vector). In some embodiments, the feature includes an acoustic feature for the input speech. In some embodiments, the device 200 is configured to digitize the received input speech and separate the input speech into features.

[0093] In some embodiments, a recognition hypothesis comprises a sequence of one or more words and an associated initial confidence score for each of the one or more words. For example, in some embodiments, a recognition hypothesis comprises at least one pairing comprising a word and an initial confidence score associated with the word.

[0094] According to some examples, the method includes: at box 406, determining a confidence score adjustment value for the word in the pair (e.g., including the corresponding word and the corresponding initial confidence score). For example, the device 200 can determine a confidence score adjustment value for the word for each word in the pair. In some embodiments, the device 200 determines the confidence score adjustment value for the word by retrieving the confidence score adjustment value from a confidence score adjustment database. The confidence score adjustment database can be configured to store confidence score adjustment values ​​for each of a plurality of words, for example. In some embodiments, the device 200 determines the confidence score adjustment value for the word by retrieving statistics associated with the word from a statistical data repository (e.g., a data store) and calculating the confidence score adjustment value for the word based at least in part on the retrieved statistics. The statistical data repository can be configured to store statistics for each of a plurality of words, for example.

[0095] According to some examples, the method includes: at block 408, generating adjusted confidence scores for the terms in the pairing. For example, the device 200 may generate an adjusted confidence score for each term in the pairing. For example, in some embodiments, the device 200 generates the adjusted confidence score for the term based at least in part on a confidence score adjustment value for the term. In some embodiments, the device 200 generates the adjusted confidence score for the term by applying the confidence score adjustment value associated with the term to the initial confidence score for the term. In some embodiments, applying the confidence score adjustment value associated with the term to the initial confidence score for the term includes increasing the initial confidence score, for example, adding the confidence score adjustment value associated with the term to the initial confidence score. Additionally or alternatively, in some embodiments, applying the confidence score adjustment value associated with the term to the initial confidence score for the term includes decreasing the initial confidence score.

[0096] According to some examples, the method includes: at box 410, determining whether the adjusted confidence score for the word meets the rejection threshold. For example, the device 200 can determine whether the adjusted confidence score for each word meets the rejection threshold for the word. In some embodiments, the rejection threshold can be different for different words. In some embodiments, the device 200 determines whether the adjusted confidence score for the word meets (e.g., exceeds, is lower than, is equal to, etc.) the rejection threshold by comparing the adjusted confidence score for the word with the rejection threshold. In some embodiments, in response to determining that at least one word in the recognition hypothesis is associated with an adjusted confidence score that fails to meet the corresponding rejection threshold, the recognition hypothesis is rejected. Additionally or alternatively, in some embodiments, in response to determining that each word in the recognition hypothesis is associated with an adjusted confidence score that meets the corresponding rejection threshold, the recognition hypothesis is accepted. In some embodiments, in response to determining that a word is associated with an adjusted confidence score that fails to meet the rejection threshold for the word, the word is rejected. Additionally or alternatively, in some embodiments, in response to determining that a term is associated with an adjusted confidence score that satisfies a rejection threshold for the term, the term is accepted.

[0097] According to some examples, the method includes, at block 412a, in response to determining that the adjusted confidence score for each word satisfies a rejection threshold, accepting the recognition hypothesis as recognized speech.

[0098] According to some examples, the method includes, at block 412b, in response to determining that the adjusted confidence score for at least one word fails to satisfy a rejection threshold, rejecting the recognition hypothesis as recognized speech.

[0099] Figure 5 A flowchart depicting example operations for adapting speech recognition confidence scores to a user according to at least one embodiment of the present disclosure is shown. Specifically, Figure 5An example process 500 is depicted. In some embodiments, process 500 is embodied by computer program code stored on a non-transitory computer-readable storage medium of a computer program product, the computer program code being configured to execute to perform the process as depicted and described. Alternatively or additionally, in some embodiments, process 500 is performed by one or more specially configured computing devices, such as apparatus 200, either alone or in communication with one or more other components, devices, systems, etc. In this regard, in some such embodiments, apparatus 200 is specifically configured by computer-coded instructions (e.g., computer program instructions) stored thereon (e.g., stored in memory 204 and / or another component depicted and / or described herein and / or otherwise accessible to apparatus 200) to perform the operations depicted and described. In some embodiments, apparatus 200 communicates with one or more external devices, systems, devices, etc. to perform one or more of the operations as depicted and described. For example, in some embodiments, apparatus 200 communicates with at least one external data repository, client system, etc. to perform one or more of the operations as depicted and described. For purposes of simplicity of description, process 500 is described as being performed by apparatus 200 and from the perspective of the apparatus.

[0100] Although the example process 500 depicts a particular sequence of operations, the sequence may be varied without departing from the scope of the present disclosure. For example, some of the depicted operations may be performed in parallel or in a different sequence that does not substantially affect the functionality of the process 500. For example, in some embodiments, the process 500 may continue from block 404 and / or may be combined with the above. Figure 4 The blocks 406 to 408 are described as being executed in parallel. In other examples, different components of the example device or system implementing process 500 may perform functions substantially simultaneously or in a particular sequence. For purposes of simplicity of description, process 500 is described as continuing from block 408.

[0101] According to some examples, the method includes, at block 502, identifying an expected response corresponding to an input speech. In some embodiments, the apparatus 200 identifies the expected response based at least in part on retrieving the expected response from a data repository. In some embodiments, the expected response includes a sequence of one or more words that the user expects to speak, for example, at a particular step in a workflow.

[0102] According to some examples, the method includes: at block 504, comparing the words in the pair associated with the recognition hypothesis generated for the input speech with the corresponding words in the expected response. For example, the device 200 may compare each word in the pair associated with the recognition hypothesis with the corresponding word in the expected response. In some embodiments, the device 200 uses a matching algorithm (e.g., such as the one described above in conjunction with Figure 3 In some embodiments, the apparatus 200 uses a matching algorithm to compare the words in the pair with the corresponding words in the expected response. In some embodiments, the apparatus 200 uses a matching algorithm to determine the corresponding word in the expected response for the corresponding word based at least in part on the position of the corresponding word in the sequence of one or more words of the recognition hypothesis.

[0103] According to some examples, the method includes, at block 506, updating statistics associated with the terms in the recognition hypothesis in response to determining that the recognition hypothesis matches the expected response. For example, for each term, the device 200 may update statistics associated with the corresponding term in response to determining that the recognition hypothesis matches the expected response. In some embodiments, the recognition hypothesis may be determined to match the expected response if each term in the recognition hypothesis matches the corresponding term in the expected response. It should be understood that in some embodiments, other criteria may be used to determine a match. For example, in some embodiments, the device 200 may determine that the recognition hypothesis matches the expected response if a certain percentage of terms match. In some embodiments, the statistics associated with a given term may be updated in response to determining that the term matches the corresponding term in the expected response. In some embodiments, an updated confidence score adjustment value for a term may be calculated using the updated statistics for the term, wherein the updated confidence score adjustment value may be configured to apply to subsequent recognition hypotheses that include another instance of the same term. In some embodiments, the statistics associated with a particular term include one or more of a sum metric, a count metric, or a sum of squares metric. In some embodiments, statistics for a particular word are determined based at least in part on each initial confidence score associated with the word in a recognition hypothesis and / or a set of initial confidence scores associated with the word in at least one historical recognition hypothesis.

[0104] In some embodiments, the confidence score adjustment value associated with a term can be updated in response to determining that the term matches a corresponding term in the expected response. For example, in some embodiments, the device 200 updates the confidence score adjustment value associated with the term in response to determining that the term matches a corresponding term in the expected response to generate an updated confidence score adjustment value for the term. In some embodiments, the confidence score adjustment value associated with the term is updated in response to determining that the recognition hypothesis matches the expected response. For example, in some embodiments, the device 200 updates the confidence score adjustment value associated with the term in the recognition hypothesis in response to determining that the expected response matches the recognition hypothesis. In some such embodiments, in response to determining that the recognition hypothesis matches the expected response, the device 200 may update the confidence score adjustment value associated with the term to generate an updated confidence score adjustment value for the term. For example, in some embodiments, in response to determining that the recognition hypothesis matches the expected response, the device 200 may update the confidence score adjustment value associated with each term to generate an updated confidence score adjustment value for each term.

[0105] In some embodiments, the device 200 uses one or more models to generate statistical data and / or updated confidence score adjustment values ​​for words. Additionally or alternatively, in some embodiments, the device 200 generates updated confidence score adjustment values ​​based at least in part on one or more conditions. For example, in some embodiments, one or more models are trained, configured, etc. to generate updated confidence score adjustment values ​​that meet one or more conditions. The one or more conditions can be configured to reduce the risk of false rejection, false acceptance, errors that exceed vocabulary insertion, and / or other factors that affect the accuracy of the speech recognition system. An example of such a condition is that the average adjusted confidence score associated with the word is at a predefined value above the rejection threshold. In this regard, one or more models can be trained, configured, etc. to generate updated confidence score adjustment values ​​for a given word so that the average adjusted confidence score for the word based on the initial confidence score and / or historical confidence score associated with the word meets the rejection threshold with a predefined value. Another example of such a condition is that the average adjusted confidence score associated with the word is N standard deviations above the rejection threshold, where N can be any value. In this regard, one or more models can be trained, configured, etc. to generate updated confidence score adjustment values ​​for a given term such that the average adjusted confidence score for the term based on the initial confidence score and / or historical confidence scores associated with the term is N standard deviations above the rejection threshold.

[0106] In some embodiments, updating the confidence score adjustment value for a given word includes: updating the confidence score adjustment database to include an updated confidence score adjustment value generated for the word, wherein the updated confidence score adjustment value can be applied to a subsequent recognition hypothesis that includes another instance of the same word. For example, in some embodiments, the device 200 is configured to apply the updated confidence score adjustment value for a given word to a subsequent recognition hypothesis that includes the word by adjusting the initial confidence score for the word in the subsequent recognition hypothesis based at least in part on the updated confidence score adjustment value. For example, the device 200 can be configured to apply the updated confidence score adjustment value to a subsequent recognition hypothesis that includes another instance of the same word and the associated initial confidence score by adjusting the initial confidence score for the word in the subsequent recognition hypothesis based at least in part on the updated confidence score adjustment value.

[0107] To provide some context, an example environment in which the example embodiments of the present disclosure can be implemented is a voice-guided work dialogue, in which a central system or computer is docked with multiple portable terminals carried or worn by workers, and a voice interface can require the workers to speak certain information, such as numerical check digits. For example, in the previously described picking application, a user can be directed to a shelf or a container and asked to pick up a product and speak two numerical check digits to indicate that they are at the appropriate shelf or container. The possible responses for a single numerical word spoken by the user are "zero" to "nine", and therefore, for two check digits, there are 100 possible combinations. The user will then speak two check digits, such as "three five" (3, 5).

[0108] Then, the system extracts features based on the two check digits spoken by the user, and generates a recognition hypothesis. According to one aspect of the present invention, the system or terminal learns that the expected response to the expected check digit of the shelf or cargo box place from the user is the word "three five". Then, the system compares the recognition hypothesis with the expected response (for example, "three five") to determine whether the recognition hypothesis matches the expected response. In one embodiment, only a single expected response is used. Alternatively, there can be several possible expected responses, which form a subset within a larger possible response set. For example, the set or subset of responses can be stored in a memory, for use in this type of manner.

[0109] In some embodiments, to improve the accuracy and efficiency of the speech recognition system, the speech recognition confidence score is adapted for the user based on the expected response. To this end, in some embodiments, the speech recognition system updates the confidence score adjustment value applied to the initial confidence score of the hypothesized word.

[0110] in conclusion

[0111] After benefiting from the teachings presented in the previous description and the associated drawings, the field to which the present disclosure belongs will think of many modifications and other embodiments of the present disclosure set forth herein. Therefore, it should be understood that the embodiments are not limited to the specific embodiments disclosed, and modifications and other embodiments are intended to be included within the scope of the appended claims. In addition, although the previous description and the associated drawings have described example embodiments in the context of certain example combinations of elements and / or functions, it should be understood that different combinations of elements and / or functions can be provided by alternative embodiments without departing from the scope of the appended claims. In this regard, for example, as can be set forth in some of the appended claims, it is also conceivable to combine elements and / or functions that are different from those explicitly described above. Although specific terms are employed herein, they are used only in a general and descriptive sense, and not for restrictive purposes.

[0112] Although an example processing system has been described above, specific implementations of the subject matter and functional operations described herein may be implemented in other types of digital electronic circuitry, or in computer software, firmware, or hardware (including the structures disclosed in this specification and their structural equivalents), or in a combination of one or more of them.

[0113] The embodiments of the themes and operations described herein can be implemented in digital electronic circuits, or in computer software, firmware or hardware (including the structures disclosed in this specification and their structural equivalents), or in a combination of one or more thereof. The embodiments of the themes described herein can be implemented as one or more computer programs (i.e., one or more modules of computer program instructions) encoded on a computer storage medium for information / data processing devices to execute or control the operation of the information / data processing devices. Alternatively or additionally, the program instructions can be encoded on an artificially generated propagation signal (e.g., a machine-generated electrical signal, optical signal or electromagnetic signal) that is generated to encode information / data for transmission to a suitable receiver device for information / data processing devices to execute. The computer storage medium can be a computer-readable storage device, a computer-readable storage substrate, a random access memory array or device or a serial access memory array or device, or a combination of one or more thereof, or be included in a computer-readable storage device, a computer-readable storage substrate, a random access memory array or device or a serial access memory array or device, or a combination of one or more thereof. Furthermore, although a computer storage medium is not a propagated signal, a computer storage medium can be a source or destination of computer program instructions encoded in an artificially generated propagated signal. A computer storage medium can also be, or be included in, one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices).

[0114] The operations described herein may be implemented as operations performed by an information / data processing apparatus on information / data stored on one or more computer-readable storage devices or received from other sources.

[0115] The term "data processing apparatus" encompasses all types of apparatus, equipment, and machines for processing data, including, for example, a programmable processor, a computer, a system on a chip, or one or a combination of the foregoing. The apparatus may include dedicated logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). In addition to hardware, the apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, a protocol stack, a repository management system, an operating system, a cross-platform runtime environment, a virtual machine, or a combination of one or more of these. The apparatus and execution environment may implement a variety of different computing model infrastructures, such as web services, distributed computing infrastructures, and grid computing infrastructures.

[0116] A computer program (also referred to as a program, software, software application, script, or code) can be written in any form of programming language (including compiled or interpreted languages, declarative languages, or procedural languages), and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program can, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or information / data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or portions of code). A computer program can be deployed to execute on one computer, or on multiple computers located at one location or distributed across multiple locations and interconnected by a communication network.

[0117] The process and logic flow described herein can be performed by one or more programmable processors that execute one or more computer programs, to perform actions by operating input information / data and generating output. For example, the processor that is suitable for executing a computer program includes any one or more processors of a general-purpose microprocessor and a special-purpose microprocessor and a digital computer of any kind. Generally speaking, the processor will receive instructions and information / data from a read-only memory or a random access memory or both. The basic element of a computer is a processor for performing actions according to instructions and one or more memories for storing instructions and data. Generally speaking, a computer will also include or be operably coupled to one or more mass storage devices (for example, magnetic disks, magneto-optical disks or optical disks) for storing data, to receive information / data from the one or more mass storage devices or to transmit information / data to the one or more mass storage devices, or both. However, a computer does not need to have such devices. Devices suitable for storing computer program instructions and information / data include all forms of non-volatile memory, media, and storage devices, including, by way of example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0118] To provide for interaction with a user, embodiments of the subject matter described herein may be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information / data to the user, as well as a keyboard and a pointing device (e.g., a mouse or trackball through which the user can provide input to the computer). Other kinds of devices may be used to provide for interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and the input from the user may be received in any form, including acoustic input, voice input, or tactile input. In addition, a computer may interact with a user by sending documents to and receiving documents from a device used by the user; for example, by sending a web page to a web browser on a user's client device in response to a request received from the web browser.

[0119] Embodiments of the subject matter described herein can be implemented in a computing system that includes a back-end component (e.g., as an information / data server), or includes a middleware component (e.g., an application server), or includes a front-end component (e.g., a client computer with a graphical user interface or web browser through which a user can interact with a specific implementation of the subject matter described herein), or any combination of one or more such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital information / data communication (e.g., a communication network). Examples of communication networks include local area networks ("LANs") and wide area networks ("WANs"), internetworks (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks).

[0120] The computing system may include a client and a server. The client and the server are usually remote from each other and typically interact through a communication network. The relationship between the client and the server is generated by means of a computer program running on a respective computer, having a client-server relationship with each other. In some embodiments, the server transmits information / data (e.g., an HTML page) to a client device (e.g., for the purpose of displaying information / data to a user interacting with the client device and receiving user input from a user interacting with the client device). The information / data generated at the client device (e.g., the result of the user interaction) can be received from the client device at the server.

[0121] Although this specification includes many specific implementation details, these details should not be interpreted as limiting the scope of any disclosure or claimed content, but rather as a description of the features of a particular disclosed embodiment. Certain features described herein in the context of a separate embodiment may also be implemented in combination in a single embodiment. On the contrary, the various features described in the context of a separate embodiment may also be implemented in multiple embodiments or in any suitable sub-combination. In addition, although features may be described above as working in certain combinations and even initially claimed, in some cases, one or more features from the claimed combination may be deleted from the combination, and the claimed combination may be for a variation of a sub-combination or sub-combination.

[0122] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that such operations be performed in the particular order shown or in a sequential order or that all of the illustrated operations be performed to achieve desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of the various system components in the embodiments described above should not be understood as requiring this separation in all embodiments, and should be understood that described program components and systems can be commonly integrated into a single software product or grouped into multiple software products.

[0123] Thus, specific embodiments of the present subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the particular order or sequential order shown to achieve the desired results. In some implementations, multitasking and parallel processing may be advantageous.

Claims

1. A computer-implemented method for adapting speech recognition to one or more users, the computer-implemented method comprising: Receiving input speech; generating a recognition hypothesis based on the input speech and using a recognition algorithm, wherein the recognition hypothesis comprises at least one pairing comprising a term and an initial confidence score associated with the term; determining whether the recognition hypothesis matches the expected response based at least in part on comparing the word to a corresponding word in the expected response; as well as In response to determining that the recognition hypothesis matches the expected response, statistics associated with the term are updated, wherein the updated statistics are configured to update a confidence score adjustment value associated with the term.

2. The computer-implemented method of claim 1 , wherein: The statistics associated with the terms include one or more of a sum metric, a count metric, or a sum of squares metric.

3. The computer-implemented method of claim 1 , wherein: The statistical data for the term is determined based at least in part on each confidence score associated with the term in the recognition hypothesis and / or each initial confidence score associated with the term in at least one historical recognition hypothesis.

4. The computer-implemented method of claim 1 , further comprising: The updated statistical data is stored in a data repository.

5. The computer-implemented method of claim 1 , further comprising: generating an adjusted confidence score for the term by applying the confidence score adjustment value for the term to an initial confidence score associated with the term; as well as A determination is made as to whether the adjusted confidence score satisfies a rejection threshold for the term.

6. The computer-implemented method of claim 5, wherein: Responsive to determining that at least one term in the recognition hypothesis is associated with an adjusted confidence score that fails to satisfy the rejection threshold for the at least one term, the recognition hypothesis is rejected.

7. The computer-implemented method of claim 5, wherein: Responsive to determining that each term in the recognition hypothesis is associated with an adjusted confidence score that satisfies a respective rejection threshold for each term, the recognition hypothesis is accepted.

8. The computer-implemented method of claim 5, further comprising: generating an updated confidence score adjustment value for the term based at least in part on the updated statistical data; as well as The updated confidence score adjustment value is applied to a subsequent recognition hypothesis that includes another instance of the same term and an associated initial confidence score by adjusting the initial confidence score for the term in the subsequent recognition hypothesis based at least in part on the updated confidence score adjustment value.

9. An apparatus for adapting speech recognition to one or more users, the apparatus comprising at least one processor and at least one non-transitory memory, the at least one non-transitory memory comprising program code stored thereon, wherein the at least one non-transitory memory and the program code are configured to: Receiving input speech; generating a recognition hypothesis based on the input speech and using a recognition algorithm, wherein the recognition hypothesis comprises at least one pairing comprising a term and an initial confidence score associated with the term; determining whether the recognition hypothesis matches the expected response based at least in part on comparing the word to a corresponding word in the expected response; and In response to determining that the recognition hypothesis matches the expected response, statistics associated with the term are updated, wherein the updated statistics are configured to update a confidence score adjustment value associated with the term.

10. A computer program product for adapting speech recognition to one or more users, the computer program product comprising at least one non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising an executable portion, the executable portion being configured to: Receiving input speech; generating a recognition hypothesis based on the input speech and using a recognition algorithm, wherein the recognition hypothesis comprises at least one pairing comprising a term and an initial confidence score associated with the term; determining whether the recognition hypothesis matches the expected response based at least in part on comparing the word to a corresponding word in the expected response; and In response to determining that the recognition hypothesis matches the expected response, statistics associated with the term are updated, wherein the updated statistics are configured to update a confidence score adjustment value associated with the term.

Images