Voice command creation device, computer-readable storage medium, and voice processing device

Abstract

A voice command creation device according to the present disclosure comprises: an initial word acquisition unit that acquires an initial word on which words of a voice command for operating a machine by voice are based; a preprocessing unit that creates a prompt for instructing generation of the words of the voice command on the basis of the initial word; a transmission unit that transmits the prompt to an interactive response device; a reception unit that receives, from the interactive response device, the words which have been generated on the basis of the initial word; and an output unit that outputs the received words.

Description

Voice command creation device, computer-readable storage medium, and voice processing device

[0001] The present disclosure relates to a voice command creation device, a computer-readable storage medium, and a voice processing device.

[0002] Conventionally, voice commands have been used for operating machines. Such machines incorporate a voice recognition engine that recognizes speech. Voice recognition engines employ methods that use statistical models, neural networks, or the like to recognize only expressions that conform to predetermined words and rules (grammar). These voice recognition engines require voice commands to be registered. A voice command associates a word, which is the content of a user's utterance, with a machine action corresponding to the word. When the voice recognition engine recognizes the utterance of a word, it causes the action corresponding to the recognized word to be executed.

[0003] Japanese Unexamined Patent Application Publication No. 2019-126902

[0004] As described above, it is necessary to register rules and words in a voice recognition engine. If a rule or word uttered by a user is not registered in the voice recognition engine, the voice command may not be executed. Although machine manufacturers create voice commands to cover all possible user utterances, it is difficult to cover every case.

[0005] A technique that assists in creating voice commands is desired.

[0006] A voice command creation device according to the present disclosure includes: an initial word acquisition unit that acquires an initial word which is a base for a word of a voice command for operating a machine by voice; a preprocessing unit that creates a prompt instructing generation of a voice command word based on the initial word; a transmission unit that transmits the prompt to an interactive response device; a reception unit that receives a word generated based on the initial word from the interactive response device; and an output unit that outputs the received word.

[0007] This diagram illustrates the configuration of voice commands. This is a block diagram of the voice command creation device according to the first embodiment. This is a diagram showing an example of a response from an interactive response device. This is a block diagram of the voice command creation device according to the second embodiment. This is a diagram showing an example of a prompt. This is a diagram showing an example of a response from an interactive response device. This is a block diagram of the voice command creation device according to the third embodiment. This is a block diagram of the voice command creation device according to the fourth embodiment. This is a diagram showing an example of a classification result. This is a diagram showing an example of a semantic classification result. This is a block diagram of the voice command creation device according to the fifth embodiment. This is a block diagram of the voice processing device. This is a hardware configuration diagram of the voice command creation device.

[0008] Embodiments of this disclosure will be described below with reference to the drawings. In the following description, components having the same or similar functions will be denoted by the same reference numerals. Duplication of these components may be omitted.

[0009] In this application, "based on XX" means "based on at least XX," and includes cases where it is based on another element in addition to XX. Furthermore, "based on XX" is not limited to cases where XX is used directly, but also includes cases where it is based on something that has been calculated or processed. "XX" is any element (for example, any information).

[0010] The voice command creation device 100 of this embodiment is applicable to information processing devices such as personal computers and mobile terminals. The type of information processing device is not particularly limited.

[0011] As a prerequisite, let's explain the structure of the voice commands. In this embodiment, the words and sentences to be recognized are pre-registered for the voice commands. Figure 1 shows an example of a voice command. In this voice command, text called "words" is associated with machine actions (operations). Voice commands and actions are provided by the machine manufacturer. The machine manufacturer creates voice commands by associating the actions of their machines with words. While voice commands with pre-registered words and sentences to be recognized offer high accuracy, they may not recognize words or sentences that are not registered. There are dialects and other regional differences in the field, and registering all of these as words would be a burden on the voice command creator.

[0012] [First Embodiment] Figure 2 is a block diagram of the voice command creation device 100 of the first embodiment. The voice command creation device 100 includes an initial word acquisition unit 1, a pre-processing unit 2, a transmission unit 3, a reception unit 4, and an output unit 5. The configuration of the voice command creation device 100 differs slightly depending on whether the interactive response device is fine-tuned or external information is used (referred to as RAG (Retrievable Augmented Generation)). The second embodiment describes an example of using external information, and the third embodiment describes an example of fine-tuning.

[0013] The initial word acquisition unit 1 acquires the initial word from the creator of the voice command. The initial word is a string of characters and serves as the basis for variations of the word.

[0014] The preprocessor 2 creates a prompt that instructs the generation of words based on the initial word. For example, if the initial word is "coolant," the preprocessor 2 creates a prompt that instructs "create words related to coolant." A word is a sentence and includes "coolant" or synonyms for "coolant."

[0015] The transmission unit 3 sends a prompt to the interactive response device. The interactive response device responds to the prompt. The interactive response device is an Artificial Intelligence (AI). The interactive response device generates text based on the input prompt.

[0016] The receiving unit 4 receives the response from the interactive response device. Figure 3 shows an example of a response from the interactive response device. The interactive response device responds with the words "Start the coolant," "Stop the coolant," "Adjust the coolant," "Check the coolant," "Refill the coolant," and "Adjust the coolant temperature."

[0017] Output unit 5 outputs the response of the interactive response device to a display screen or the like. Output unit 5 outputs words generated based on the initial word. The creator of the voice command creates the voice command by associating words with actions, using the output words as a reference.

[0018] The voice command creation device 100 receives input from the voice command creator for initial words that will serve as the basis for creating words, and instructs the interactive response device to create words related to the initial words. The voice command creator associates machine actions with the words created by the interactive response device to create a voice command. The speech recognition in this embodiment is a speech recognition that recognizes words that have been registered in advance. In such speech recognition, words that are not registered will not be recognized. Therefore, the voice command must include all the words spoken by the machine user. If there are any missing words, the machine will not operate. However, anticipating all of the user's utterances is a burden on the voice command creator. The voice command creation device 100 reduces the burden on the voice command creator by generating word candidates using the interactive response device.

[0019] [Second Embodiment] Figure 4 is a block diagram of the voice command creation device 100 of the second embodiment. The voice command creation device 100 of the second embodiment includes an initial word acquisition unit 1, a preprocessing unit 2, a transmission unit 3, a receiving unit 4, an output unit 5, a storage unit 6, and a search unit 7.

[0020] The memory unit 6 stores specialized knowledge about the machine. General-purpose interactive response devices do not have sufficient specialized knowledge. Specialized knowledge includes at least one of the machine's screen name, device name, part name, and operation name. If the machine is a machine tool, it also includes at least one of the machine's mode name, parameter name, G-code name, signal name, and operation method name. As specialized knowledge, the memory unit 6 stores, for example, a dictionary of technical terms, a dictionary of synonyms / related terms for technical terms, a dictionary of related terms for technical terms, the machine's manual, and the machine's operation history. Specialized knowledge may also include the operation history of voice commands. Such word information is used to update voice commands.

[0021] The search unit 7 searches the memory unit 6 for terms related to the initial word. Related terms include synonyms, synonyms, superordinate terms, subordinate terms, and related terms. Synonyms are terms that have different forms but similar meanings. Synonyms are terms that have different forms but almost the same meaning. Superordinate terms are terms that represent higher-level concepts. Subordinate terms are terms that represent lower-level concepts. Related terms are a group of terms that have a looser relationship than synonyms or synonyms. In this embodiment, terms related to "coolant" include "cutting fluid," "shower," "center coolant," and "side coolant."

[0022] The preprocessor 2 acquires terms related to the initial word and creates a prompt that instructs the generation of a word. The prompt includes the acquired term and an instruction that instructs the creation of a word for a voice command using this term. Figure 5 shows an example of a prompt. This prompt consists of #Instruction "Create a word for a voice command that includes "coolant"", #Constraint "Create as many word candidates as possible by appropriately adding particles and verbs to the following terms", "[coolant], coolant, cutting fluid, shower, center coolant, side coolant".

[0023] The transmitting unit 3 sends a prompt to the interactive response device. The receiving unit 4 receives the response from the interactive response device, and the output unit 5 outputs the received response to a display screen or the like. Figure 6 shows an example of a response from the interactive response device. In the example in Figure 6, 42 words are created in the interactive response device.

[0024] As described above, the voice command creation device 100 of the second embodiment stores specialized knowledge related to the machine, searches for terms related to the initial word, and instructs the interactive response device to generate words based on the searched terms. In this way, by adding phrases related to the initial word to the prompt, the types of words that can be generated are broadened.

[0025] [Third Embodiment] Figure 7 is a block diagram of the voice command creation device 100 of the third embodiment. The voice command creation device 100 of the third embodiment includes an initial word acquisition unit 1, a preprocessing unit 2, a transmission unit 3, a receiving unit 4, an output unit 5, a storage unit 6, and an additional learning data generation unit 8.

[0026] The additional training data generation unit 8 generates additional training data for fine tuning. This additional training data may include a dictionary of technical terms, a machine manual, or a machine operation history. The additional training data is not particularly limited.

[0027] The interactive response device is fine-tuned using additional training data provided by the voice command generation device 100. The interactive response device learns specialized knowledge as knowledge.

[0028] The preprocessing unit 2 creates a prompt that instructs the generation of a word based on the initial word. The transmission unit 3 sends the prompt to the interactive response device. The interactive response device generates a response based on the prompt. The interactive response device of the third embodiment has learned specialized knowledge. Therefore, it provides a response based on that specialized knowledge. The receiving unit 4 receives the response from the interactive response device.

[0029] Output unit 5 outputs the response of the interactive response device to a display screen or the like. Output unit 5 presents a word generated based on the initial word. This word reflects specialized knowledge.

[0030] As described above, the voice command creation device 100 of the third embodiment generates additional learning data for learning specialized knowledge about the machine and outputs it to the interactive response device. The interactive response device generates a response based on the specialized knowledge.

[0031] [Fourth Embodiment] Figure 8 is a block diagram of the voice command creation device 100 of the fourth embodiment. The voice command creation device 100 of the fourth embodiment includes an initial word acquisition unit 1, a preprocessing unit 2, a transmission unit 3, a receiving unit 4, an output unit 5, a classification unit 9, and an element selection unit 10.

[0032] The classification unit 9 grammatically classifies the words responded to by the interactive response device. Figure 9 is an example of the classification results. The example in Figure 9 is a grammatical classification of the 21 words on the left side of Figure 6, from "start the coolant" to "check the side coolant". In the example in Figure 9, the words are classified into [object], [particle], [verb 1], and [verb 2]. The [object] tag lists the "object" of the particle "を". Following [object] is the particle "を", which expresses the semantic relationship between [object] and the verb. Following the particle "を" are [verb 1] and [verb 2]. [Verb 1] and [verb 2] together constitute one sa-verb. The sa-verb exemplified here is a compound word formed by adding "する" to a Chinese noun. If the 21 words on the right side of Figure 6 are classified, they are classified into [object], [particle], and [verb 1]. In addition, if the word is "Get the coolant out," then "Verb 1" and "Verb 2" can also be expressed in the imperative form by combining the conjunctive form of the verb "dashi" and the auxiliary verb "te."

[0033] The classification unit 9 may further classify the grammatically classified words semantically. For example, the verbs in Figure 9 are "start," "stop," "adjust," "confirm," "replenish," and "supply." These verbs can be semantically classified into "Verb 1 Start," "Verb 2 Stop," "Verb 1 Confirm," "Verb 1 Adjust," and "Verb 1 Replenish," as shown in Figure 10. "Verb 1 Start" is classified as Verb 1 "start," "Verb 1 Stop" as Verb 1 "stop," "Verb 1 Confirm" as Verb 1 "confirm," "Verb 1 Adjust" as Verb 1 "adjust," and "Verb 1 Replenish" as Verb 1 "supply" and "replenish." Further semantically classifying grammatically classified words makes it easier to associate them with actions.

[0034] The element selection unit 10 accepts the selection of elements included in the classification result items. When elements are selected, words can be created by combining the selected elements. For example, in the example in Figure 9, if two elements "coolant" and "cutting fluid" are selected from [target] and one element "start" is selected from [verb 1], the words "start the coolant" and "start the cutting fluid" are created by combining the selected elements.

[0035] According to the fourth embodiment, grammatically classifying words makes them easier to see and verify. If all possible words are listed, the number of words becomes enormous, making it difficult to confirm whether all words are covered. By classifying the phrases that make up words, word management becomes easier. If all possible words are listed, the number of words becomes enormous, and associating an action with each of them is time-consuming and carries a high risk of errors. By classifying words grammatically and specifying words by combining the classified elements, it becomes easier to associate them with actions.

[0036] [Fifth Embodiment] Figure 11 shows a voice command creation device 100 according to the fifth embodiment. The voice command creation device 100 according to the fifth embodiment includes an initial word acquisition unit 1, a pre-processing unit 2, a transmission unit 3, a reception unit 4, an output unit 5, and a language specification unit 11.

[0037] The language specification unit 11 accepts the language specification for the word. The preprocessing unit 2 adds the language specification when creating a prompt that instructs the generation of the word. The transmission unit 3 sends the prompt to the interactive response device. The interactive response device responds with a word related to the initial word in the specified language. The receiving unit 4 receives the response from the interactive response device.

[0038] The voice command creation device 100 of the fifth embodiment supports other languages. The voice command of the fifth embodiment may also include the classification unit 9 and element selection unit 10 of the fourth embodiment. The classification unit 9 grammatically classifies words in other languages ​​as well. For example, "start the coolant" would be classified as follows: [verb] "start", "begin", "initiate", [article] "the", [object] "coolant", "cooling fluid", "cooling agent", "center coolant", "side coolant".

[0039] The element selection unit 10 accepts the selection of elements from each category and accepts combinations of selected elements. Users can create voice commands by associating combinations of elements with actions.

[0040] The interactive response device is equipped with a translation function. The interactive response device is assumed to have already learned the grammatical rules of each language. Therefore, by adding the prompt "grammatically divide", the interactive response device can grammatically classify texts other than Japanese. The classification unit 9 may further semantically classify the words of other languages ​​that have been grammatically classified.

[0041] [Sixth Embodiment] FIG. 12 is a block diagram of a speech processing apparatus 200 that operates a machine tool using speech recognition. The speech processing apparatus 200 comprises a database 21 in which speech commands are registered, and a recognition engine 22 that recognizes speech registered in the database. The database 21 has speech commands registered therein. Each speech command is associated with a word generated by the above-described speech command creation apparatus 100 and an action (operation) of the machine tool. The speech processing apparatus 200 is built in or attached to a machine tool or a numerical controller for a machine tool. The speech processing apparatus 200 acquires an operator's speech. When the acquired speech matches a word registered in the database 21, the recognition engine 22 causes the numerical controller to execute the action associated with the recognized word. Here, it is assumed that the words registered in the database 21 are created by the speech command creation apparatuses according to the first to fifth embodiments.

[0042] Hereinafter, the hardware configuration of the speech command creation apparatus 100 to which the present disclosure is applied will be described. FIG. 13 is a hardware configuration diagram of the speech command creation apparatus 100. As an example, the speech command creation apparatus 100 comprises a processor 111, a main storage device (memory) 112, an auxiliary storage device (memory) 113, a network interface 114, and a device interface 115, and is implemented as a computer in which these components are connected via a bus 116.

[0043] Although the speech command creation apparatus 100 in FIG. 13 includes one of each constituent element, it may include a plurality of the same constituent elements.

[0044] The processor 111 may be an electronic circuit (processing circuit, Processing circuitry, CPU, GPU, FPGA, ASIC, etc.) that performs at least one of computer control and arithmetic operation. Furthermore, the processor 111 may be any of a general-purpose processor, a dedicated processing circuit designed to execute specific operations, or a semiconductor device including both a general-purpose processor and a dedicated processing circuit.

[0045] The main memory 112 may store instructions executed by the processor 111 and various data, and the information stored in the main memory 112 may be read by the processor 111. The auxiliary memory 113 is a memory device other than the main memory 112.

[0046] The network interface 114 is an interface for connecting to a communication network by wire or wireless connection. The communication network may be a WAN (Wide Area Network), LAN (Local Area Network), PAN (Personal Area Network), or a combination thereof. The voice command creation device 100 exchanges information with the external device 117 via the network interface 114. The device interface 115 is an interface such as USB for direct connection to the external device 118. The external devices 117 and 118 may be personal computers, tablet terminals, or smartphones. The external devices 117 and 118 may also be storage devices.

[0047] While embodiments of this disclosure have been described in detail above, this disclosure is not limited to the individual embodiments described above. These embodiments can be added, replaced, modified, partially deleted, etc., in any way that does not depart from the spirit of the invention or from the idea and intent of this disclosure derived from the claims and their equivalents. For example, the order of operations and processes in the embodiments described above are shown as examples only and are not limited thereto. The same applies when numerical values ​​or mathematical formulas are used in the description of the embodiments described above.

[0048] The following are annotations relating to embodiments of the present disclosure. (Annotation 1) A voice command creation device (100) according to one aspect of the present disclosure includes: an initial word acquisition unit (1) that acquires initial words that will be the basis of words for voice commands used to operate a machine by voice; a pre-processing unit (2) that creates a prompt instructing the generation of words for voice commands based on the initial words; a transmission unit (3) that transmits the prompt to an interactive response device; a reception unit (4) that receives words generated from the interactive response device based on the initial words; and an output unit (5) that outputs the received words. (Annotation 2) The voice command creation device (100) includes a storage unit (6) that stores specialized knowledge; a search unit (7) that searches the storage unit for terms related to the initial words; and the pre-processing unit (2) creates a prompt instructing the generation of words for voice commands based on terms searched by the search unit. (Note 3) The voice command creation device (100) includes a storage unit (6) for storing specialized knowledge and an additional learning data generation unit (8) for generating additional learning data for the interactive response device based on the specialized knowledge. (Note 4) The storage unit (6) includes at least one of the machine's screen name, device name, and operation name. (Note 5) The machine is a numerical control device, and the storage unit (6) includes at least one of the machine's screen name, device name, part name, operation name, machine mode name, parameter name, G code name, signal name, and operation method name. (Note 6) The voice command creation device (100) includes a classification unit (9) for grammatically classifying words received from the interactive response device, and the output unit (5) outputs the classification results. (Note 7) The classification unit (9) further classifies the grammatically classified words semantically. (Note 8) The voice command creation device (100) includes an element selection unit (10) that accepts the selection of word elements classified by the classification unit and combinations of the selected elements, and creates a voice command by accepting the association between the combination of elements and an action. (Note 9) The voice command creation device (100) includes a language specification unit (11) that accepts the specification of a language, and the preprocessing unit (2) creates a prompt that instructs the generation of a word in the specified language.(Supplementary Note 10) A computer-readable storage medium (112, 113) according to an aspect of the present disclosure stores a program that causes a computer to operate as: an initial word acquisition unit (1) that acquires an initial word serving as a basis for words of a voice command for operating a machine by voice; a preprocessing unit (2) that creates a prompt instructing generation of a voice command word based on the initial word; a transmission unit (3) that transmits the prompt to an interactive response device; a reception unit (4) that receives a word generated based on the initial word from the interactive response device; and an output unit (5) that outputs the received word. (Supplementary Note 11) A voice processing device (200) according to an aspect of the present disclosure includes a recognition engine (22) that recognizes voice using a database (21) in which words created by any one of the voice command creation devices (100) according to claims 1 to 9 are registered, and operates a machine tool according to the recognized voice.

[0049] 100 Voice command creation device 1 Initial word acquisition unit 2 Preprocessing unit 3 Transmission unit 4 Reception unit 5 Output unit 6 Storage unit 7 Search unit 8 Additional learning data generation unit 9 Classification unit 10 Element selection unit 200 Voice processing device 21 Database 22 Recognition engine 111 Processor 112 Main storage device 113 Auxiliary storage device

Claims

1. A voice command creation device comprising: an initial word acquisition unit that acquires initial words to be used as the basis for voice commands to operate a machine by voice; a pre-processing unit that creates a prompt that instructs the generation of voice command words based on the initial words; a transmission unit that transmits the prompt to an interactive response device; a receiving unit that receives words generated from the interactive response device based on the initial words; and an output unit that outputs the received words.

2. The voice command creation device according to claim 1, comprising: a storage unit for storing specialized knowledge; and a search unit for searching the storage unit for terms related to the initial word, wherein the preprocessing unit creates a prompt that instructs the search unit to generate the words of a voice command based on the terms it has searched.

3. The voice command creation device according to claim 1, comprising: a memory unit for storing specialized knowledge; and an additional learning data generation unit for generating additional learning data for the interactive response device based on the specialized knowledge.

4. The voice command creation device according to claim 2 or 3, wherein the storage unit includes at least one of the screen name, device name, and operation name of the machine.

5. The voice command creation device according to claim 2 or 3, wherein the machine is a numerical control device, and the storage unit includes at least one of the machine's screen name, device name, component name, operation name, machine mode name, parameter name, G code name, signal name, and operation method name.

6. The voice command creation device according to claim 1, further comprising a classification unit that grammatically classifies words received from the interactive response device, and an output unit that outputs the classification result.

7. The voice command creation device according to claim 6, wherein the classification unit further classifies the grammatically classified words semantically.

8. A voice command creation device according to claim 6 or 7, comprising an element selection unit that selects elements of words classified by the classification unit and accepts combinations of the selected elements, and creating a voice command by accepting associations between the combinations of elements and actions.

9. The voice command creation device according to claim 1, comprising a language specification unit that accepts language specification, wherein the preprocessing unit creates a prompt that instructs the generation of a word in the specified language.

10. A computer-readable storage medium containing a program that causes the computer to operate as follows: an initial word acquisition unit that acquires initial words to be used as the basis for voice commands to operate a machine by voice; a pre-processing unit that creates a prompt to instruct the generation of voice command words based on the initial words; a transmission unit that transmits the prompt to an interactive response device; a reception unit that receives words generated from the interactive response device based on the initial words; and an output unit that outputs the received words.

11. A voice processing device comprising a voice operation unit that recognizes speech using a database in which words created by any one of the voice command creation devices described in claims 1 to 9 are registered, and operates a machine tool in accordance with the recognized speech.

Images