Methods for operating a voice control system and voice control system

Abstract

Method for operating a voice control system, in which, before the start of an evaluation process, a. speech expressions are captured by means of at least one microphone (1), b. the captured speech expressions are converted into a storable, digital audio format using a first control device (2), c. by means of a second control device (3) an indexing of the digitized speech expressions and / or parts thereof with a start time, an end time and status information is carried out, d. the generated index information is stored in a first memory (4), e. the indexed language expressions are stored as language blocks in a second memory (5), wherein this memory (5) is implemented as a ring buffer, f. following the start of the evaluation process: the language blocks stored in the ring buffer (5) are compared with predefined language blocks stored in a third buffer (7) using a third control device (6) and g. a predefined action is performed when a match is detected.

Description

[0001] The invention relates to a method for operating a voice control system and to a voice control system.

[0002] Modern technical devices nowadays offer an ever-increasing range of functions. To nevertheless enable the simplest and most intuitive operation possible, voice control systems are being used more and more frequently.

[0003] Voice control systems typically consist of at least one microphone and at least one control device. The microphone captures any speech expression. These speech expressions are compared with speech expressions stored in a memory. When a match is detected, an action associated with the speech expression is triggered. Such an action could be, for example, executing a function or playing back speech through a loudspeaker.

[0004] Many voice control systems used today operate on the basis of single-word recognition, which must be activated by manually starting the system. In this case, the voice commands are not connected sentences but individual words. If the voice control system is used, for example, to operate a navigation system in a vehicle, the voice commands are typically city or street names used for address entry.

[0005] While single-word voice control has already reached a high level of maturity, a dialogue-oriented, natural language-based voice control system is often desired. This requires not only the capture and evaluation of explicit, individual speech expressions, but also speech expressions contained within longer speech sequences.

[0006] A solution for capturing, storing, and evaluating voice commands in a voice control system is disclosed in German patent application DE 100 30 369 A1. This document describes a method that enables improved recognition of voice input. It proposes a shift register for buffering speech expressions. If the user presses the start button for the voice control system after having already uttered relevant speech expressions, the buffered sequences can be included in the evaluation. However, the necessary method for processing the captured speech expressions is not disclosed in the cited document. Rather, the invention describes only a buffering process lasting a few milliseconds before the start button is pressed.

[0007] Document US 2016 / 0189716A1 describes a configuration consisting of multiple processors and memory elements. In this configuration, one of the two processors and its associated memory continuously captures speech expressions. The second processor, along with its corresponding memory chip, is solely responsible for recognizing a startup password. Once this password is recognized, the information in the second memory is evaluated and converted into executable instructions.

[0008] US patent 2005 / 0071170 A1 proposes a speech recognition engine that enables the recognition of voice commands and voice data. For example, it can distinguish between the command "make a phone call" and the input of contact information, especially a phone number.

[0009] Document US 2017 / 0068513 A1 proposes a digital assistant, similar to those now found in smartphones, which can recognize user commands as they are being entered. Once the input meets certain criteria, particularly a keyword, the cached speech data is analyzed and made available to the digital assistant.

[0010] A similar principle underlies DE 10 2009 059792 A1, in which speech recognition is only carried out for already cached speech inputs once a switching element is manually operated.

[0011] The task remains to describe a high-performance method for capturing, storing and processing speech in a voice control system, as well as a suitable device for carrying out the method.

[0012] According to the invention, this problem is solved by a method according to claim 1 and a device according to claim 9. Advantageous embodiments of the invention are described in the dependent claims.

[0013] The technical effect achievable with the disclosed solution is the ability to capture and evaluate speech expressions to a significantly greater extent. This makes it possible to upgrade speech control systems, which currently often rely solely on single-word recognition, to the processing of natural language.

[0014] The method according to the invention provides that, prior to the start of an evaluation process, speech expressions are first captured by one or more microphones. A speech expression is understood to be any sequence of words in the form of human speech uttered by the operator of the voice control system. In accordance with the nature of human language, such speech expressions are in particular words that are causally related, i.e., usually uttered in the form of sentences. However, it is also conceivable that periods of silence, in which no speech is given, or incomplete sentences or sentence fragments are captured. Periods of silence occur in particular between individual words or sentence fragments.However, phases in which no speech is given due to other influences, such as the necessary concentration on the traffic situation, are also part of a speech expression within the meaning of the disclosed invention.

[0015] An operator is understood to be any person whose speech expressions are captured and evaluated during the process. In a motor vehicle, this could be, in particular, the driver or the other vehicle occupants. The disclosed process should, in principle, be designed such that all speech expressions, regardless of their origin, can be processed as speech expressions to be evaluated.

[0016] The speech expressions captured by the microphone(s) are initially in analog form. To store them on a digital storage medium, they must be converted into a digital format. The inventive method provides for this conversion of the analog speech expressions into coherent, digital speech blocks. Such a speech block is understood to be a coherent speech expression of a defined length, which exists in digital form.

[0017] The length of each speech block is determined by the control device. According to the invention, these speech blocks can have a fixed or variable length. A fixed length for the speech blocks is particularly advantageous when longer periods of silence need to be filtered out of the speech expressions. This is especially important with regard to limiting the required memory space.

[0018] During the conversion of analog speech expressions into storable digital speech blocks, the individual blocks are indexed. This means that additional information, in particular a start time, an end time, and status information, is added to each speech block. In an advantageous embodiment of the invention, the index information generated is converted into a tabular form and stored in a first memory location. The digitized speech content, i.e., the speech blocks, are stored in a second memory location.

[0019] The second memory for storing the speech blocks is designed as a ring buffer according to the invention. A ring buffer is a storage technology in which the oldest memory content can be replaced by the newest memory content after an initial filling. This logic is ensured by a corresponding software application in a control device.

[0020] In a further step of the inventive method, the additional information stored in tabular form and the stored language blocks are evaluated. For this purpose, the contents of the language blocks are compared with the stored language blocks.

[0021] According to an advantageous embodiment of the inventive method, the evaluation process can be initiated by pressing a mechanical button or by speaking a keyword, a so-called wake-up word. A continuous, background-running automatic evaluation of the speech blocks, combined with a search for predefined keywords, is also a possible evaluation logic. For example, it is conceivable to filter the word "telephone" and its variations, such as "telephone call" or "telephone conversation," from the speech expressions and use them as a start signal for the further processing of the described method.

[0022] Following the start of the evaluation process, the language blocks are read from memory and compared with language blocks stored in the voice control system. If a match is found, an action associated with the respective language block is triggered.

[0023] In advantageous embodiments of the method, the triggered actions are operating functions and / or information outputs. In a motor vehicle, for example, the operating request "Please increase the temperature" can be implemented by executing the actual function and subsequently outputting the information "Temperature is increased".

[0024] In a further advantageous embodiment of the described method, the voice control system is used in a means of transportation, e.g., a motor vehicle. The actions that can be triggered can be either related to the means of transportation or not. Actions not related to the means of transportation include, for example, searching for a telephone number or a shopping opportunity in the immediate vicinity.

[0025] In an advantageous embodiment of the described method, the control devices provided for carrying out the described process steps are combined and designed as an integrated control device.

[0026] In a preferred embodiment of the described method, the storage elements necessary for storing the information are physically combined, with the control of the individual storage elements being carried out using a software application.

[0027] If no match is found despite activation of the voice control system, an advantageous embodiment of the method according to the invention provides for the possibility of a system-based prompt. For example, the system can ask the user to repeat their request.

[0028] In a further advantageous embodiment of the method according to the invention, the speech blocks are not stored locally, e.g., in a memory located in the motor vehicle, but in a cloud storage system accessed via a data connection. The advantage of this embodiment is the potentially larger storage capacity.

[0029] The invention further relates to a device with which speech can be analyzed and predefined actions can be executed depending on the recognized speech content. According to the invention, the device comprises at least one microphone, at least one control device, and at least one memory.

[0030] With the help of at least one microphone, speech expressions can be captured, especially those uttered by a user of the voice control system.

[0031] Furthermore, according to the invention, a first control device is provided in which a software application for converting the analog signals of the microphone into storable, digital audio formats is implemented. This first control device can be implemented as a separate component in the voice control system. A design in which such a control device with the associated software is integrated into one of the microphones is also a solution within the meaning of the invention.

[0032] In addition to the first control device, a second control device is part of the disclosed device, and here too, a solution in which both control devices are combined in a single unit represents a suitable embodiment within the meaning of the invention. A software solution is implemented in the second control device, which can be used to index the digitized speech expressions. Indexed speech expressions are referred to as speech blocks within the meaning of the invention.

[0033] The process of indexing involves supplementing language expressions with descriptive properties. These include, in particular, the start and end times, as well as status information.

[0034] A first memory is provided for storing the index information. The digital language blocks are stored in a second memory, which may also be physically combined with the first memory in a common storage unit.

[0035] The second memory for storing the language blocks is implemented as a ring buffer in the disclosed device. This refers to a storage technique in which a memory is initially filled with information. Once the capacity limit is reached, the oldest content is deleted to allow for the storage of further content. If deleting one content is insufficient, further content is deleted until sufficient storage capacity is available.

[0036] To enable the evaluation of the speech blocks after storage, the device according to the invention provides a third control device. This control device incorporates a software application that allows the stored speech blocks to be compared with speech blocks stored in a third memory location. If the control device detects a match during such a comparison, a predefined action is executed.

[0037] In an advantageous embodiment of the disclosed invention, the device is used in a means of transport, in particular a motor vehicle.

[0038] In a further advantageous embodiment of the disclosed invention, the action is a control function and / or an information output. This could, for example, be increasing the indoor temperature or the nearest restaurant requested by the operator.

[0039] In an advantageous embodiment of the disclosed device, the first and / or second and / or third control device is combined and designed as an integrated control device.

[0040] In a further advantageous embodiment of the disclosed device, the first and / or second and / or third storage units necessary for storing the information are physically combined, with the control of the individual storage units being carried out using a software application.

[0041] In a further advantageous embodiment of the device, if no match can be found between the captured and processed speech expressions and the predefined, stored speech expressions, a predefined query is executed. This could, for example, be the general query "I'm sorry, I didn't understand you. Please repeat your request." However, a specific query such as "Unfortunately, I can't find the location you're looking for. Did you perhaps mean Hamburg as your navigation destination?" is also a solution within the scope of the invention.

[0042] In an advantageous embodiment of the disclosed device, the recorded language blocks, the stored language blocks and / or the index information are optionally stored in a cloud storage device.

[0043] The device and method are described below using an exemplary embodiment with reference to the Fig. 1 and Fig. 2 explained. Fig. Figure 3 illustrates the indexing of the digitized language expressions.

[0044] In Fig. Figure 1 shows an exemplary setup of the voice control system. It consists of a single microphone 1 for capturing speech expressions. A first control device 2 is provided for converting the speech expressions into a storable, digital audio format. Additionally, a second control device 3 is provided, which is used to index the speech expressions.

[0045] The process of indexing language expressions is in Fig. Section 3 explains this. Arbitrary speech expressions consisting of speech and silence components are shown here. These speech expressions are processed using a control device. The information obtained in this process, in particular a start time, an end time, and status information, is stored in Table 11.

[0046] In the illustrated embodiment of the invention, a first memory 4 is provided as a storage element. The generated index information can be stored in tabular form in this first memory 4.

[0047] The previously indexed language blocks can be stored in a second memory location, number 5. This memory location is implemented as a ring buffer.

[0048] A further control device 6 implements a software application for evaluating the captured speech expressions. If this evaluation detects a match between the captured speech block and the stored speech blocks, an action can be triggered using a third control device 6. In the exemplary embodiment, the action is a functional execution of an operating system 8 or an audio output via a loudspeaker 9.

[0049] The process of the inventive method for operating a voice control system is described in Fig. 2 illustrated in an exemplary design.

[0050] In this process, the operator's speech expressions are first captured in a first process step 100 using one or more microphones. The speech expressions are initially in analog form. In the exemplary embodiment, the speech expressions are converted into a storable digital file format in a process step 200 using a first control device 2. In a process step 300, the resulting digital information is divided into individual speech blocks. This process step is carried out using a second control device 3 and software located on it. During the division of the speech expressions, the necessary index information (start time, end time, and / or processing status) is also generated using the second control device 3.Both the language blocks and the index information are stored (step 400), with the second memory 5 being designed as a ring buffer for storing the language blocks.

[0051] For evaluation, in process step 500 the stored language blocks are read and compared with language blocks stored in a third memory location 7 (step 600). The processing of the individual language blocks is controlled by the stored index information.

[0052] If a match is found between the recorded and stored language blocks and the stored language blocks, an associated action is triggered in a process step 700. When using the voice control system in a motor vehicle, the action could, for example, be a function in an operating system 8 or an audio output via a loudspeaker 9. Reference symbol list 1. Microphone 2. First control device 3. Second control device 4. First storage 5. Second storage (ring storage) 6. Third control device 7. Third storage 8. Operating system 9. Speaker 10. Language expressions 11. Index table 100 Capture 200 Digitize 300 Convert and Index 400 saves 500 readouts 600 comparisons 700 Triggering a control signal

Claims

[1] Method for operating a voice control system in which, before the start of an evaluation process a. speech expressions are captured by means of at least one microphone (1), b. the captured speech expressions are converted into a storable, digital audio format using a first control device (2), c. by means of a second control device (3) an indexing of the digitized speech expressions and / or parts thereof with a start time, an end time and status information is carried out, d. the generated index information is stored in a first memory (4), e. the indexed language expressions are stored as language blocks in a second memory (5), wherein this memory (5) is implemented as a ring buffer, f. following the start of the evaluation process: the language blocks stored in the ring buffer (5) are compared with predefined language blocks stored in a third buffer (7) using a third control device (6) and g. a predefined action is performed when a match is detected. [2] Method according to claim 1, wherein the start of the evaluation process is initiated by at least one of the group of: - Pressing a mechanical button, - Speaking a keyword, and - Automatic, permanent evaluation of language blocks based on defined keywords. [3] Method according to one of claims 1 and 2, characterized by that the action performed is an operating function and / or an information output. [4] Method according to any one of the preceding claims, characterized bythat the voice control system is used in a means of transportation. [5] Method according to any one of the preceding claims, characterized by , that the first and / or second and / or third control device (2, 3, 6) are integrated into a control device. [6] Method according to any one of the preceding claims, characterized by , that the first and / or second and / or third memory (4, 5, 7) are physically combined in one memory unit. [7] Method according to any of the preceding claims, characterized by , that the voice control system can issue a predefined query in the event that no match can be found between the indexed and stored speech expressions and the speech expressions stored in the third memory (7). [8] Method according to any one of the preceding claims, characterized by, that the first and / or the second and / or the third storage (4, 5, 7) are implemented as cloud storage. [9] Device for operating a voice control system, comprising a. at least one microphone (1) for recording speech expressions, b. at least one first control device (2) for converting the captured speech expressions into a storable, digital audio format, c. at least a second control device (3) for indexing the captured speech expressions by means of a start time, an end time and a processing status, d. a first memory (4) for storing the generated index information, e. a second memory (5) for storing the indexed language expressions as language blocks, wherein the memory (5) for storing the language blocks is designed as a ring buffer, f. a control device (6) for comparing the speech blocks stored in the ring buffer (5) with speech blocks stored in a third buffer (7) following the start of an evaluation process, wherein a predefined action is executed when a match is detected. [10] Device according to claim 9, characterized by that the action performed is an operating function and / or an information output. [11] Device according to one of claims 9 or 10, characterized by that the device can be used in a means of transport. [12] Device according to any one of claims 9 to 11, characterized by , that the first and / or second and / or third control device (2, 3, 6) are combined into an integrated control device. [13] Device according to any one of claims 9 to 12, characterized by, that the first and / or second and / or third memory (4, 5, 7) are physically combined in an integrated memory unit. [14] Device according to any one of claims 9 to 13, characterized by , that a predefined query can be issued by means of the voice control system in the event that no match can be found between the indexed speech blocks stored in a second memory (5) and the speech blocks stored in a third memory (5). [15] Device according to any one of claims 9 to 14, characterized by , that the first and / or the second and / or the third storage (4, 5, 7) are implemented as cloud storage.

Images