Information processing device, information processing method, and program

Abstract

The present disclosure relates to an information processing device, an information processing method, and a program capable of achieving more suitable input of a voice command. In the present disclosure, a function specification unit specifies a function based on understanding of the intent of speech in a voice command given by a user, and a command evaluation unit outputs an alternative command for executing the function instead of the voice command on the basis of an evaluation of the voice command. The technology according to the present disclosure can be applied, for example, to a device that implements various functions on the basis of a voice command given by a user.

Description

Information processing device, information processing method, and program

[0001] The present disclosure relates to an information processing device, an information processing method, and a program, and more particularly to an information processing device, an information processing method, and a program that enable more suitable input of voice commands.

[0002] In recent years, advances in speech recognition technology and language processing technology have made it possible to realize a certain function of a voice-operated system by using various voice commands. However, if the command is not appropriately used, it may be mistaken for a function that the user does not want.

[0003] Patent Document 1 discloses a tool that controls the clarification of voice commands and the resolution of misrecognition errors by visually displaying multiple interpretations of a voice command on a display.

[0004] Furthermore, Patent Document 2 describes a technique for improving the accuracy of command recognition by deleting other phrases that are classified as the same function as the voice command from the command list.

[0005] JP 2006-146182 A JP 2007-226098 A

[0006] Conventionally, a user has not been able to know which of the various ways to phrase a voice command is more appropriate.

[0007] The present disclosure has been made in view of such circumstances, and aims to enable more suitable input of voice commands.

[0008] The information processing device of the present disclosure is an information processing device that includes a function identification unit that identifies a function based on understanding the voice intent of a voice command given by a user, and a command evaluation unit that outputs an alternative command for executing the function in place of the voice command based on an evaluation of the voice command.

[0009] The information processing method disclosed herein is an information processing method in which an information processing device identifies a function based on understanding the voice intent of a voice command given by a user, and outputs an alternative command for executing the function in place of the voice command based on an evaluation of the voice command.

[0010] The program disclosed herein is a program for causing a computer to execute a process of identifying a function based on understanding the voice intent of a voice command given by a user, and outputting an alternative command for executing the function in place of the voice command based on an evaluation of the voice command.

[0011] In the present disclosure, a function is identified based on understanding the voice intent of a voice command given by a user, and an alternative command for executing the function in place of the voice command is output based on an evaluation of the voice command.

[0012] 1 is a diagram illustrating an overview of the technology according to the present disclosure. FIG. 1 is a block diagram illustrating an example of a configuration of a voice operation system according to the present disclosure. FIG. 2 is a flowchart illustrating the operation of the voice operation system. FIG. 3 is a flowchart illustrating command evaluation and voice synthesis processing. FIG. 4 is a diagram illustrating an example of command evaluation. FIG. 5 is a diagram illustrating an example of command evaluation based on sound similarity. FIG. 6 is a diagram illustrating an example of the structure of a Seq2Seq model. FIG. 7 is a diagram illustrating an example of a function implemented by a voice command. FIG. 8 is a diagram illustrating an example of a voice recognition result for a voice command. FIG. 9 is a diagram illustrating an example of intent understanding of a voice command. FIG. 10 is a diagram illustrating an example of command candidates. FIG. 11 is a diagram illustrating an example of other function commands. FIG. 12 is a diagram illustrating an example of similarity with other function commands. FIG. 13 is a diagram illustrating an example score for ease of voice intent understanding. FIG. 14 is a diagram illustrating an example score for command conciseness. FIG. 15 is a diagram illustrating an example score for user vocabulary knowledge. FIG. 16 is a diagram illustrating an example of final scores for a voice command and a command candidate. FIG. 17 is a diagram illustrating an example of a voice command that is difficult to recognize. FIG. 18 is a diagram illustrating an example of a voice command that is difficult to recognize. FIG. 19 is a diagram illustrating another example of command evaluation based on sound similarity. FIG. 19 is a diagram illustrating an example of the hardware configuration of a computer.

[0013] Modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described below in the following order.

[0014] 1. Overview of the technology according to the present disclosure 2. Configuration and operation of a voice operation system 3. Example of score calculation 4. Specific example of command evaluation 5. Modification 6. Example of computer hardware configuration

[0015] 1. Overview of the Technology Relating to the Present Disclosure> In recent years, advances in speech recognition technology and language processing technology have made it possible to implement a certain function of a voice-operated system, etc., using various voice commands. For example, to implement a function for setting the volume of a television receiver (TV) to level 3, a user can issue commands such as "Set the volume to 3," "Set the volume to 3," or "Turn the volume down to 3."

[0016] However, users have not been able to know which of various phrases for a voice command is more appropriate. Specifically, even if a voice command is used with an inappropriate phrase, the user is not aware of this. Furthermore, in such a case, a method for providing feedback to the user on a more appropriate phrase for the voice command has not been considered.

[0017] In contrast, the technology disclosed herein defines more appropriate voice commands based on two criteria: "ease of system recognition" and "ease of use for the user." First, when a user issues a voice command to implement a certain function, command candidates that can implement that function are automatically generated. Then, the generated command candidates are evaluated based on the criteria, and the more appropriate command is recommended to the user.

[0018] The "ease of recognizing a system" is evaluated based on the similarity (or dissimilarity) of the sound information of commands that realize other functions. For example, the similarity can be found by converting command candidates generated based on the voice command given by the user into a sound vector representation and then calculating the similarity with the vector representation of commands that realize other functions.

[0019] "User usability" is evaluated based on the length of the command and the user's vocabulary knowledge. For example, as shown in FIG. 1 , when user U1 instructs "Turn the volume down to 3" to set the volume of television D2 to 3, command candidates for setting the volume to 3 are generated, such as "Turn the volume down to 3," "Turn the volume down to 3," and "Turn the volume down to 3." Of these, "Turn the volume down to 3" is more verbose than "Turn the volume down to 3," resulting in a longer command that is less user-friendly for the user. Furthermore, since user U1 instructs "Turn the volume down to 3," if "volume" is easier for the user to say than "volume," it is considered that "Turn the volume down to 3" is more appropriate than "Turn the volume down to 3." Therefore, as shown in FIG. 1 , the function for setting the volume to 3 is executed, and "Turn the volume down to 3" is recommended (feedback) as a more appropriate command.

[0020] According to the technology disclosed herein, the user can learn more appropriate commands and can use voice commands that are easier for the system to recognize and use from the next time onwards, thereby improving the user experience.

[0021] In recent years, the technology of instructing machines using natural language commands has made remarkable progress, as seen in the rise of language models such as GPT (Generative Pretrained Transformer). The technology disclosed herein is a technology that enables a user to evaluate a voice command provided by the user and recommend or provide guidance on a better command, and may be useful in future computing systems that can interact with users.

[0022] 2. Configuration and Operation of Voice Operation System (Configuration of Voice Operation System) FIG. 2 is a block diagram showing an example configuration of a voice operation system according to an embodiment of the technology disclosed herein.

[0023] 2 can be configured as a device that realizes various functions based on voice commands given by a user. For example, the voice operation system 1 can be configured as a television receiver (television) that turns the power on / off and sets the channel and volume based on voice commands given by the user.

[0024] The voice operation system 1 is configured to include an information processing device 10 , a microphone 20 , and a speaker 30 .

[0025] The information processing device 10 is configured as a computer including a processor such as a CPU (Central Processing Unit), a memory, an HDD (Hard Disk Drive), an SSD (Solid State Drive), and the like.

[0026] The microphone 20 (hereinafter referred to as the mic 20) converts sound into an electrical signal and supplies it to the information processing device 10. The mic 20 may be an electronic component built into the voice-operated system 1 (device), or may be an acoustic device externally attached to the voice-operated system 1 (device).

[0027] The speaker 30 converts an electrical signal supplied from the information processing device 10 into sound and outputs the sound. The speaker 30 may also be an electronic component built into the voice-operated system 1 (device), or may be an acoustic device externally attached to the voice-operated system 1 (device).

[0028] When a voice command is given by the user via the microphone 20, the information processing device 10 executes a function desired by the user (desired function) and outputs an alternative command for executing the desired function in place of the voice command via the speaker 30. The desired function may be any of the functions executed by any device.

[0029] The information processing device 10 is composed of a voice activity detection unit 110, a voice intention understanding unit 120, a user vocabulary database 130, a device function execution unit 140, a command generation unit 150, a command evaluation unit 160, and a voice synthesis unit 170. At least some of the functional blocks constituting the information processing device 10 are realized by a CPU executing a predetermined program.

[0030] The voice activity detection unit 110 detects only the voice activity of the user from the electrical signal (sound) supplied from the microphone 20. Here, a time period in which the volume (power) of the sound is equal to or greater than a predetermined threshold is detected as a voice command and supplied to the voice intention understanding unit 120.

[0031] The voice intention understanding unit 120 is configured as a function identification unit that identifies a function desired by the user (desired function) by identifying intention information that indicates the user's intention based on voice intention understanding of the voice command detected by the voice segment detection unit 110. The intention information corresponding to the desired function is supplied to the device function execution unit 140 and the command generation unit 150.

[0032] The voice intention understanding unit 120 has a voice recognition module 210 that recognizes a voice command as a voice, and an intention understanding module 220 that understands the intention of the recognized voice command. That is, the voice intention understanding unit 120 performs voice recognition of the voice command using the voice recognition module 210, and then uses the intention understanding module 220 to understand the intention of the voice command based on the voice recognition result, thereby identifying the desired function.

[0033] The speech recognition module 210 is composed of a preprocessing section and a speech recognition model such as a Seq2Seq model. The preprocessing section generates acoustic features from speech. The speech recognition model converts the generated acoustic features into higher-order features using an encoder, and generates words one by one using a decoder based on the higher-order features and previously output word sequences. The generated words are stored in the user vocabulary database 130 as the user's vocabulary knowledge.

[0034] The intent understanding module 220 is composed of a language model. The language model may include a language model (generative AI (Artificial Intelligence) model) constructed using a Transformer such as BART or GPT. The intent understanding module 220 is pre-trained using large amounts of text data. Specifically, the intent understanding module 220 is fine-tuned using command sentences as input and semantic analysis expressions as target outputs. The semantic analysis expression can be expressed by an intent indicating "which function" and a slot indicating the parameters for that function, and can be a sequence of intent types, slot types, and values. For example, if the input is ("volume", "3", "to", "do"), the output can be ("[intent:set_volume", "[slot:volume", "3", "]", "]").

[0035] Based on the intention information from the voice intention understanding unit 120, the device function execution unit 140 executes a device function (desired function) corresponding to the intention information.

[0036] The command generation unit 150 generates command candidates to be candidates for alternative commands based on the desired function corresponding to the intention information from the voice intention understanding unit 120 .

[0037] The command generation unit 150 is configured with a language model. The language model may include a large-scale language model (generative AI model) such as GPT. For example, if the description of the set_volume(volume) function (function), one of the device's functions, is "set the volume to 'volume'," and set_volume(3) is provided as intent information from the voice intention understanding unit 120, a representative command such as "set the volume to 3" is obtained. Based on the obtained representative command, command candidates are obtained by inputting "list 10 voice commands that 'set the volume to 3'" into the large-scale language model. The obtained command candidates are provided to the command evaluation unit 160 along with the representative command corresponding to the desired function (the original voice command provided by the user). Note that both the representative command and the command candidates are text data. Furthermore, the representative command does not need to exactly match the voice command provided by the user.

[0038] The command evaluation unit 160 outputs an alternative command for executing a desired function in place of the voice command based on an evaluation of the original voice command given by the user. Specifically, the command evaluation unit 160 evaluates the original voice command and command candidates from the command generation unit 150, and outputs the alternative command to the voice synthesis unit 170.

[0039] The voice synthesis unit 170 generates voice feedback for the user by voice synthesizing the alternative command output from the command evaluation unit 160. The generated voice feedback is output via the speaker 30.

[0040] The speech synthesis unit 170 is composed of a preprocessing unit, an acoustic model such as a Seq2Seq model, and a neural vocoder model such as WaveNet. The preprocessing unit converts the text of the alternative command into a label string of phonemes and accents. The acoustic model generates acoustic features from the label string. The neural vocoder model converts the acoustic features into an electrical signal waveform.

[0041] (Operation of the Voice Operation System) The operation of the voice operation system 1 of Fig. 2 will be described with reference to the flowchart of Fig. 3. The process of Fig. 3 starts when the voice operation function of the voice operation system 1 is enabled, for example.

[0042] In step S11, the information processing device 10 determines whether or not a user's voice has been input based on an electrical signal from the microphone 20. Step S11 is repeated until a user's voice has been input, and when it is determined that a user's voice has been input, the process proceeds to step S12.

[0043] In step S12 , the voice activity detector 110 identifies a voice activity from the electrical signal from the microphone 20 .

[0044] In step S13 , the speech intent understanding unit 120 identifies intent information from the speech in the speech section detected by the speech section detection unit 110 .

[0045] In step S14 , the device function execution unit 140 executes the device function (desired function) corresponding to the intention information identified by the voice intention understanding unit 120 .

[0046] In step S15, the command generation unit 150 generates command candidates that are candidates for alternative commands for executing the desired function, based on the intention information identified by the voice intention understanding unit 120.

[0047] In step S16, the command evaluation unit 160 and the voice synthesis unit 170 execute command evaluation and voice synthesis processing. After that, the process returns to step S11, and steps S11 to S16 are repeated every time the user's voice is input.

[0048] (Command Evaluation and Voice Synthesis Processing) The command evaluation and voice synthesis processing executed in step S16 of FIG. 3 will be described in detail with reference to the flowchart of FIG.

[0049] In step S21, the command evaluation unit 160 evaluates the ease of recognition (ease of understanding the voice intent) of the system (voice operation system 1) for the original voice command given by the user and each of the candidate commands generated by the command generation unit 150.

[0050] In step S22, the command evaluation unit 160 evaluates the ease of use (user usability) for the original voice command and each of the command candidates.

[0051] In step S23, the command evaluation unit 160 calculates a final score for each command based on the scores obtained by evaluating the ease of understanding the voice intent and the usability for the user for each of the original voice command and the command candidate commands.

[0052] In step S24, the command evaluation unit 160 determines whether the maximum score of the command candidate is higher than the score of the original voice command. If it is determined that the maximum score of the command candidate is higher than the score of the original voice command, the process proceeds to step S25.

[0053] In step S25, the voice synthesis unit 170 generates voice feedback by voice synthesizing the command candidate with the highest score as an alternative command, and thereby outputs voice feedback (alternative command) for executing the desired function in place of the voice command via the speaker 30.

[0054] On the other hand, if it is determined in step S24 that the maximum score of the command candidate is not higher than the score of the original voice command, i.e., if the maximum score of the command candidate is lower than the score of the original voice command, step S25 is skipped, i.e., the voice command given by the user is considered to be the optimal voice command, and no voice feedback like an alternative command is output.

[0055] According to the above process, instead of the voice command given by the user, an alternative command that is easy to understand the voice intent and is easy for the user to use is fed back to the user, so that the user can use a voice command that is easily recognized by the system and easy for the user to use from the next time onwards. In other words, it is possible to input a more suitable voice command, and as a result, the user experience can be improved.

[0056] 3. Example of Score Calculation> Here, an example of calculation of the score of each command in the command evaluation described above will be described.

[0057] (Calculation of the system's ease of recognition score) The system's ease of recognition (ease of understanding the voice intent) can be evaluated by whether the voice intent understanding unit can easily distinguish the command to be evaluated from other function commands that may execute other functions different from the desired function.

[0058] A simple method, as shown in Figure 5, involves performing speech synthesis on all command candidates generated by a command generation unit, calling a speech intent understanding unit for the resulting synthetic speech, and using the prediction probability (reliability) of the speech recognition module and intent understanding module as a score. However, with this method, the sound quality of the synthetic speech may have a significant impact on the score. If the sound quality of the synthetic speech is insufficient, the prediction probability of the speech recognition module will be unduly reduced. In addition, the computational cost of performing speech synthesis and speech recognition on all command candidates is high.

[0059] In contrast, the technology disclosed herein can evaluate the ease of recognition of a system based on the sound similarity between each of the commands to be evaluated (original voice command and command candidate) and other function commands that may execute functions other than the desired function. The sound similarity is expressed by the similarity between vector expressions of phoneme strings into which commands are converted using a DNN (Deep Neural Network) model, without using synthetic speech.

[0060] Specifically, as shown in FIG. 6, each command that results in the same function is converted into a vector representation of a phoneme string using a DNN model, and the similarity with the vector representation of other function commands is calculated.

[0061] Here, let one of the command candidates be c, and let c' be a group of other function commands that realize a different function from command candidate c. Then, the system's recognition ease score, Score system (c) can be defined by the following equation (1).

[0062]

[0063] In equation (1), v(d) represents the vector representation of command d, and sim(v, v') represents the similarity between vector representations v and v'. In other words, the smaller the similarity, the better the command candidate.

[0064] In the example of FIG. 6, two command candidates “Turn down the volume to 3” and “Set the volume to 3” are respectively represented by vector expressions v １ , v ２ and the two other function commands "Turn down the volume by 3" and "Volume down by 3" are respectively expressed as vectors v ３ , v ４ In FIG. 6, the vector representation v １ and the vector representation v ３ The similarity sim(v １ , v ３ ) is 0.8, and the vector representation v １ and the vector representation v ４ The similarity sim(v １ , v ４ ) is set to 0.3. Also, the vector expression v ２ and the vector representation v ３ The similarity sim(v ２ , v ３ ) is 0.5, and the vector representation v ２ and the vector representation v ４ The similarity sim(v ２ , v ４ ) is set to 0.3. In the example of FIG. 6, the vector representation v ２ would be a better command candidate.

[0065] A possible method for converting a command into a vector representation of a phoneme string (in other words, a vector representation that can distinguish a phoneme string) is to use a Seq2Seq model. A possible training method for the Seq2Seq model is to use a word string as input and a phoneme string as output.

[0066] As shown in FIG. 7, the Seq2Seq model converts a word sequence into a fixed-length vector representation v ｅ and the decoder DEC converts it into a vector representation v ｅ The phoneme sequence is decoded from the input. The output of this encoder ENC can be used to calculate the sound similarity between commands. Through training, commands with similar phoneme sequences are expected to have similar vector representations.

[0067] (User Ease of Use Score Calculation) User ease of use (user usability) can be evaluated by the simplicity of commands and the user's vocabulary knowledge.

[0068] The length of the sound can be used to measure the conciseness of a command. The length of the sound can be estimated from the number of moras. A mora is a segmental unit of sound with a certain time length. Basically, one mora corresponds to one kana character, but two yo-on sounds are considered to be one mora (for example, "onryo" (volume) is four moras), and a glottal stop is considered to be one independent mora. If the number of moras is m, the score for the conciseness of the command is: len (c) is expressed as the reciprocal of the number of moras, as in the following equation (2).

[0069]

[0070] Words that are not in the user's vocabulary knowledge are considered difficult for the user to use. The speech recognition module 210 stores content words (nouns, verbs, adjectives, adverbs) in the user vocabulary database 130 from the speech recognition results obtained for speech commands given by the user in the past. Then, a value based on the presence or absence of content words stored in the user vocabulary database 130 (whether or not the content words are included in the command to be evaluated) is used as a score for the user's vocabulary knowledge. When a command candidate c contains L content words (w １ , w ２ , ..., w Ｌ ) the score for the user's vocabulary knowledge, Score vocab (c) can be defined by the following equation (3).

[0071]

[0072] In formula (3), the content word w ｉ is not included in the vocabulary v given by the user in the past, isin(w ｉ , v) is set to 0. Content word w ｉ is in the vocabulary ν, then isin(w ｉ , ν) is (1-γ) ｓ Let s be the content word w ｉ is the number of steps since it was last used, and γ is a decay rate, e.g., 0.05. ｉ , v) weights the most recently used vocabulary.

[0073] From the above, the user usability score is user (c) is expressed by the following equation (4).

[0074]

[0075] The final score Score(c) for the command candidate c is expressed by the following equation (5).

[0076]

[0077] In addition, α and β in equations (4) and (5) are weighting parameters, respectively.

[0078] 4. Specific Examples of Command Evaluation (Examples of Commands for a Television) A specific example of command evaluation when the voice operation system 1 is configured as a television will be described.

[0079] FIG. 8 is a diagram showing an example of television functions that are realized by voice commands.

[0080] As shown in Figure 8, the function (function) for turning on the power is represented by power_on(), and the function for turning on the power is represented by power_off(). The function for setting the channel to channel is represented by set_channel(channel), and the function for setting the volume to volume is represented by set_volume(volume). In addition, the function for increasing the volume is represented by increase_volume(volume), and the function for decreasing the volume is represented by decrease_volume(volume).

[0081] Here, if the user gives a voice command such as "Turn down the volume to 3," the voice recognition module 210 of the voice intention understanding unit 120 recognizes the voice command as speech, and a voice recognition result such as that shown in FIG. 9 is obtained.

[0082] In the example of Figure 9, the speech recognition results show that the probability (reliability) that the voice command given to the user is "Turn the volume down to 3" is 0.4, and the probability (reliability) that the voice command is "Turn the volume down by 3" is 0.3.

[0083] Next, the intent understanding module 220 of the speech intent understanding unit 120 performs intent understanding on the obtained speech recognition result, thereby obtaining the intent understanding result shown in FIG.

[0084] 10, the probability (reliability) of the intent understanding (function) obtained from the speech recognition result of "Turn the volume down to 3" is set_volume(3) is 0.9, and the probability (reliability) of the intent understanding (function) obtained from the speech recognition result of "Turn the volume down to 3" is decrease_volume(3) is 0.1. Similarly, the probability (reliability) of the intent understanding (function) obtained from the speech recognition result of "Turn the volume down to 3" is decrease_volume(3) is 0.9, and the probability (reliability) of the intent understanding (function) obtained from the speech recognition result of "Turn the volume down to 3" is increase_volume(3) is 0.05.

[0085] 10, set_volume(3) (a function for setting the volume to 3) is most likely to be executed by the device function execution unit 140. On the other hand, decrease_volume(3) (a function for decreasing the volume by 3) is also likely to be executed by the device function execution unit 140.

[0086] Thereafter, the command generating unit 150 generates commands such as those shown in FIG. 11 as command candidates that result in set_volume(3).

[0087] In the example of FIG. 11, the original voice command c is "Turn down the volume to 3." ０ For example, the command candidate c is "Volume 3" １ , "Set the volume to 3" command candidate c ２ , "Set volume to 3" command candidate c ３ is generated.

[0088] These original voice commands c ０ and command candidate c １ ~c ３ (Hereinafter, the evaluation target command c ０ ~c ３ The command evaluation unit 160 evaluates the ease of recognition of the original voice command c ０ The system's recognizability is evaluated based on the similarity between the command and other functional commands that realize different functions.

[0089] Original voice command c ０As examples of functions different from the above, the two functions decrease_volume(3) and increase_volume(3) obtained as the top predicted functions in the intent understanding results of Fig. 10 are taken as examples. In this case, other function commands such as those shown in Fig. 12 are generated.

[0090] In the example of FIG. 12, in response to decrease_volume(3) (a function for decreasing the volume by 3), another function command c of "decrease the volume by 3" is ４ , "Volume 3 Down" and other function commands c ５ , "Turn down the volume by 3" and other function commands c ６ Similarly, for increase_volume(3) (a function to increase the volume by 3), another function command c of "increase the volume by 3" is generated. ７ , "Increase the volume by three" and other function commands c ８ , "Turn up the volume by 3" and other function commands c ９ is generated.

[0091] Command c to be evaluated ０ ~c ３ The system's recognition ability for the vector representation v(c ０ ) to v(c ３ ) for each vector representation v(c ４ ) to v(c ９ ) is evaluated by the similarity with the vector expression v(c ０ ), the similarity is calculated as shown in FIG.

[0092] In the example of FIG. 13, the evaluation target command c ０ Regarding other function commands c ４ The similarity with other function command c is 0.8. ５ The similarity with other function command c is 0.3. ６ The similarity with other function command c is 0.3. ７ The similarity with other function command c is 0.5. ８ The similarity with other function command c is 0.3. ９ The similarity between them is calculated to be 0.2.

[0093] Command c to be evaluated ０ and other function commands４ are similar as phoneme sequences, so the similarity is high. From the above formula (1), the maximum similarity is calculated as the system's recognition ease score, Score system (c ０ ) and Score system (c ０ ) = -0.8. １ ~c ３ Similarly, the similarity is calculated for the target command c ０ ~c ３ The system's recognizability score for each system As a result, the scores shown in FIG. 14 are obtained.

[0094] In the example of FIG. 14, the command c to be evaluated ０ Score system is -0.8, and the command to be evaluated is c １ Score system is -0.5, and the command to be evaluated is c ２ Score system is -0.45, and the command to be evaluated is c ３ Score system is obtained as -0.5.

[0095] Next, the command to be evaluated, c ０ ~c ３ The user's ease of use is evaluated.

[0096] First, the command's conciseness score len is expressed using the number of moras. For example, the command to be evaluated, c ０ The sound segmentation is "o|n|ryo|u|wo|sa|n|ni|sa|ge|te", so the command to be evaluated is c ０ The number of moras is 11, and the Score len (c ０ ) = 1 / 11 = 0.09. １ ~c ３ Similarly, the number of moras is calculated for the target command c １ ~c ３Score for each command's conciseness len As a result, the scores shown in FIG. 15 are obtained.

[0097] In the example of FIG. 15, the command c to be evaluated ０ Score len is 1 / 11=0.09, and the target command c １ Score len is 1 / 9=0.11, and the target command c ２ Score len is 1 / 14=0.07, and the target command c ３ Score len is obtained as 1 / 9 = 0.11.

[0098] Next, the user's vocabulary knowledge score, Score vocab is calculated. ０ includes the content words "volume", "3", and "lower", which are stored in the user vocabulary database 130 as the vocabulary v from 0 steps ago. If the user vocabulary database 130 does not include any other vocabulary words, the command c to be evaluated ０ ~c ３ Score for each user's vocabulary knowledge vocab As a result, the scores shown in FIG. 16 are obtained.

[0099] In the example of FIG. 16, the command c ０ Score vocab is 3 / 3=1, and the command to be evaluated is c １ Score vocab is 2 / 3=0.67, and the command to be evaluated is c ２ Score vocab is 2 / 3=0.67, and the command to be evaluated is c ３ Score vocab is obtained as 1 / 3 = 0.33.

[0100] When the weighting parameters α and β are set to α=5 and β=3, the evaluation target command c ０~c ３ The final score for each is shown in FIG.

[0101] In the example of FIG. 17, the command c ０ The final score for the command c is −0.95. １ The final score for the command c is −0.28. ２ The final score for the command c is −0.33. ３ The final score for is obtained as -0.62.

[0102] In FIG. 17, the command candidate c that has the highest final score is １ The score (-0.28) is the same as the original voice command c ０ Since the score is larger than the score of the command candidate c (-0.95), １ "Volume 3" is recommended to the user as an alternative command.

[0103] (Examples of commands that are difficult to recognize) The above describes the use of the sound similarity between commands as an index for evaluating the ease of recognition by the system. For voice commands that are difficult for the system to recognize, it is desirable to recommend alternative commands with a low sound similarity.

[0104] 18 and 19 are diagrams illustrating examples of commands that are difficult to recognize.

[0105] FIG. 18 shows voice commands for a music playback system capable of playing music, including "skip music" for implementing the function next_music() for transitioning from the currently playing music to the next music, and "stop music" for implementing the function stop_music() for stopping the currently playing music.

[0106] The commands "skip song" and "stop song" shown in Figure 18 may be difficult for the system to recognize. In this case, for example, "next song," which has a low sound similarity to "stop song," may be recommended as an alternative command for "skip song." Also, for "stop song," "stop song," which has a low sound similarity to "skip song," may be recommended as an alternative command.

[0107] FIG. 19 shows voice commands for a home automation system that performs various controls in a home, such as "Turn on the lights" which implements the function light_power_on() to turn on the power to the lights, and "Turn on the weather" which implements the function show_weather() to display a weather information program on a television.

[0108] 19, the commands "Turn on the lights" and "Turn on the weather" may be difficult for the system to recognize. In this case, for example, for "Turn on the lights," "Turn on the light" may be recommended as an alternative command, since it has a low similarity in sound to "Turn on the weather."

[0109] 5. Modifications Modifications of the above-described embodiment will be described below.

[0110] (Variation 1) In the above-described embodiment, the sound similarity is represented by the similarity between vector expressions of phoneme strings into which commands are converted using a DNN model. However, the sound similarity is not limited to this, and may be represented by the edit distance between phoneme strings of commands.

[0111] Specifically, as shown in Fig. 20, the edit distance between phoneme strings of each command that result in the same function is calculated. The edit distance is defined as the minimum number of steps required to transform one string into another by inserting, deleting, or substituting one character.

[0112] Here, let one of the command candidates be c, and let c' be a group of other function commands that realize a different function from command candidate c. Then, the system's recognition ease score, Score system (c) can be defined by the following equation (6).

[0113]

[0114] In equation (6), p(c) represents the phoneme string of command candidate c, and dist(p(c), p(c')) represents the edit distance between the phoneme strings of command candidates c and c'. In other words, the larger the edit distance, the better the command candidate.

[0115] In the example of FIG. 20 , two command candidates, "Turn the volume down to 3" and "Set the volume to 3," are converted into phoneme strings p1 and p2, respectively, and two other function commands, "Turn the volume down by 3" and "Volume down by 3," are converted into phoneme strings p3 and p4, respectively. In FIG. 20 , the edit distance dist(p1, p3) between the phoneme strings p1 and p3 is 2, and the edit distance dist(p1, p4) between the phoneme strings p1 and p4 is 7. Furthermore, the edit distance dist(p2, p3) between the phoneme strings p2 and p3 is 4, and the edit distance dist(p2, p4) between the phoneme strings p2 and p4 is 6. In the example of FIG. 20 , "Set the volume to 3," converted into the phoneme string p2, is a better command candidate because it has a larger minimum edit distance.

[0116] (Modification 2) When an alternative command is recommended, the reason for the recommendation may be fed back in addition to the alternative command. An example of a dialogue between the user and the system is shown below.

[0117] User: "Turn the volume to 3." System: "For a more concise command, you can say 'Turn the volume to 3.'"

[0118] In this example, the reason for recommending the alternative command "Set volume to 3" is that the command is more concise.

[0119] Furthermore, when recommending an alternative command, in addition to the alternative command, feedback may be provided as to which command the voice command given by the user is difficult to distinguish from. An example of a dialogue between the user and the system is shown below.

[0120] User: "Turn the volume down to 3." System: "To distinguish from 'Turn the volume down by 3,' you can also say 'Set the volume to 3.'"

[0121] In this example, it is shown that the voice command given by the user "Turn the volume down to 3" is difficult to distinguish from "Turn the volume down by 3".

[0122] (Modification 3) Instead of recommending one alternative command, two or more command candidates may be recommended as alternative commands in descending order of score. An example of a dialogue between the user and the system is shown below.

[0123] User: "Turn the volume down to 3." System: "You can say 'Turn the volume to 3' or 'Set the volume to 3'"

[0124] In this example, the top two command candidates c with the highest scores in FIG. １ , c ２ is fed back as an alternative command.

[0125] (Modification 4) It may be possible to correct erroneous recognition of a voice command given by a user. The above-described embodiment aims to improve recognition of subsequent voice commands, and is premised on successful recognition of the current voice command.

[0126] If, for example, the voice intention understanding unit 120 erroneously recognizes set_volume(3) as decrease_volume(3) in recognizing the current voice command, not only will the incorrect function be executed, but an alternative command for executing decrease_volume(3) will be recommended contrary to the user's intention. In this case, if the user denies the command, a new alternative command may be recommended based on set_volume(3), which has the next highest predicted probability in the voice intention understanding by the voice intention understanding unit 120. An example of a dialogue between the user and the system is shown below.

[0127] User: "Turn the volume down to 3." System: "Decreased the volume by 3" (runs decrease_volume(3)) System: "We recommend saying 'Decrease the volume by 3'" User: "No." System: "To set the volume to 3, you can say 'Set volume to 3'"

[0128] In this example, contrary to the user's intention to set the volume to 3, a function to lower the volume by 3 is executed. When the user denies this by saying "No," a new alternative command to set the volume to 3 that is in line with the user's intention is recommended.

[0129] (Variation 5) A history of previously recommended alternative commands may be recorded, which makes it possible to avoid recommending an alternative command that has been recommended to a user but has not been used by the user to execute the function.

[0130] (Variation 6) The user's vocabulary knowledge, which is one index for evaluating "user ease of use," may be supplemented with the degree to which content words included in the vocabulary knowledge are used in general spoken language.

[0131] (Variation 7) Ease of pronunciation may be added to the index for evaluating "user usability." Ease of pronunciation may be, for example, a quantitative index disclosed in "Sato Yoshiya, Saito Daisuke, Minematsu Nobuaki, "Study on Automatic Evaluation of Ease of Pronunciation of Japanese Words Focusing on Articulatory Movement and Acoustic Features," Research Report Music Information Science (MUS), 2019-MUS-123(27), 1-6(2019-06-15), 2188-8752."

[0132] (Modification 8) Voice intent understanding, command generation, and command evaluation may be performed based on the connection status of the device and the attributes of the user.

[0133] For example, in a home automation system capable of executing functions "light_on" and "light_off" for turning lights on and off and functions "power_strip_on" and "power_strip_off" for turning a power strip on and off, let us consider a situation in which both the lights and the power strip are connected. In this case, the following interaction between the user and the system can be realized.

[0134] User: "Turn off the lights." System: "We recommend saying 'Turn off the lights.'"

[0135] Furthermore, similarly to the above-described second modification, in addition to the alternative command, the reason for its recommendation may be fed back.

[0136] User: "Turn off the lights." System: "Currently, you have lights and a power strip connected, so to distinguish between them, we recommend saying 'Turn off the lights.'"

[0137] In this way, in a situation where both the lights and the power strip are connected, in response to the user's intention to turn off the lights, the function "light_off" for turning off the lights and the function "power_strip_off" for turning off the power strip are easily distinguished from each other by providing feedback of the alternative command "Turn off the lights."

[0138] (Variation 9) For example, JP2020-530580A discloses a system that uses shortcut command phrases to cause an automated assistant to perform various actions. In such an interactive system, when a user-defined new command is registered by text or voice, applying the technology according to the present disclosure makes it possible to recommend and register better commands.

[0139] 6. Example of Computer Hardware Configuration The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the program constituting the software is installed from a program recording medium into a computer incorporated in dedicated hardware, a general-purpose personal computer, or the like.

[0140] 21 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes using a program. The above-described information processing device 10 can be configured, for example, by a computer having a configuration similar to that shown in FIG.

[0141] The CPU 501 , ROM (Read Only Memory) 502 , and RAM 503 are interconnected by a bus 504 .

[0142] An input / output interface 505 is also connected to the bus 504. An input unit 506 including a keyboard, a mouse, etc., and an output unit 507 including a display, a speaker, etc. are connected to the input / output interface 505. Also connected to the input / output interface 505 are a storage unit 508 including a hard disk, a nonvolatile memory, etc., a communication unit 509 including a network interface, etc., and a drive 510 that drives removable media 511.

[0143] In a computer configured as described above, the CPU 501 performs the above-described series of processes by, for example, loading a program stored in the memory unit 508 into the RAM 503 via the input / output interface 505 and the bus 504 and executing it.

[0144] The program executed by the CPU 501 is installed in the storage unit 508 by being recorded on, for example, removable media 511 or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting.

[0145] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0146] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0147] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0148] The embodiments of the present disclosure are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present disclosure.

[0149] For example, the embodiment of the present disclosure can be configured as a cloud computing system in which a single function is shared and processed collaboratively by multiple devices via a network.

[0150] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by a plurality of devices.

[0151] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices.

[0152] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0153] Furthermore, the technology disclosed herein may have the following configurations: (1) An information processing device comprising: a function identification unit that identifies a function based on understanding the voice intent of a voice command given by a user; and a command evaluation unit that outputs an alternative command for executing the function in place of the voice command based on an evaluation of the voice command. (2) The information processing device described in (1), further comprising: a command generation unit that generates command candidates as candidates for the alternative command based on the identified function, wherein the command evaluation unit outputs the alternative command by evaluating the voice command and the command candidates. (3) The information processing device described in (2), wherein the command evaluation unit outputs the alternative command by calculating scores for the voice command and the command candidates based on ease of understanding the voice intent and usability for the user. (4) The information processing device described in (3), wherein the command evaluation unit outputs the command candidate with the score higher than that of the voice command as the alternative command. (5) The information processing device according to (3) or (4), wherein the command evaluation unit outputs the command candidate having the highest score from among the command candidates as the alternative command. (6) The information processing device according to any of (3) to (5), wherein the score includes a sound similarity between the voice command and each of the command candidates and an other function command that can execute another function different from the function. (7) The information processing device according to (6), wherein the sound similarity is represented by a similarity between vector expressions of phoneme strings into which commands are converted using a DNN (Deep Neural Network) model. (8) The information processing device according to (6), wherein the sound similarity is represented by an edit distance between phoneme strings of commands. (9) The information processing device according to any of (3) to (8), wherein the score includes conciseness of the command and vocabulary knowledge of the user. (10) The information processing device according to (9), wherein the conciseness of the command is represented by the reciprocal of the number of moras of speech.(11) The information processing device according to (9), wherein the user's vocabulary knowledge is represented by a value based on the presence or absence of content words that constitute the voice command given by the user in the past. (12) The information processing device according to any of (2) to (11), wherein the command generation unit generates the command candidates using a language model. (13) The information processing device according to any of (1) to (12), wherein the function identification unit identifies the function by understanding the intent of the voice command using a language model. (14) The information processing device according to (13), wherein the function identification unit understands the intent of the voice command based on a voice recognition result of the voice command using a voice recognition model. (15) The information processing device according to any of (1) to (14), further comprising a voice synthesis unit that generates voice feedback to the user by voice synthesizing the alternative command. (16) The information processing device according to any of (1) to (15), wherein the function is any of functions executed by an arbitrary device. (17) An information processing method in which an information processing device specifies a function based on understanding the voice intent of a voice command given by a user, and outputs an alternative command for executing the function in place of the voice command based on an evaluation of the voice command. (18) A program for causing a computer to execute a process in which:

[0154] REFERENCE SIGNS LIST 1 Voice operation system, 10 Information processing device, 20 Microphone, 30 Speaker, 110 Voice activity detection unit, 120 Voice intention understanding unit, 130 User vocabulary database, 140 Device function execution unit, 150 Command generation unit, 160 Command evaluation unit, 170 Voice synthesis unit, 210 Voice recognition module, 220 Intention understanding module

Claims

1. An information processing device comprising: a function identification unit that identifies a function based on understanding the voice intent of a voice command given by a user; and a command evaluation unit that outputs an alternative command for executing the function in place of the voice command based on an evaluation of the voice command.

2. The information processing device of claim 1, further comprising a command generation unit that generates command candidates as candidates for the alternative command based on the identified function, and the command evaluation unit outputs the alternative command by evaluating the voice command and the command candidates.

3. The information processing device according to claim 2, wherein the command evaluation unit outputs the alternative command by calculating a score for each of the voice command and the command candidate based on the ease of understanding the voice intent and the usability of the user.

4. The information processing device according to claim 3, wherein the command evaluation unit outputs, from among the command candidates, a command candidate having a score greater than that of the voice command as the alternative command.

5. The information processing device according to claim 3, wherein the command evaluation unit outputs, from among the command candidates, the command candidate having the maximum score as the alternative command.

6. The information processing device according to claim 3, wherein the score includes a degree of sound similarity between each of the voice command and the command candidate and a different function command that can execute a different function from the function.

7. The information processing device according to claim 6, wherein the sound similarity is represented by a similarity between vector expressions of phoneme strings converted from commands using a DNN (Deep Neural Network) model.

8. The information processing device according to claim 6, wherein the sound similarity is represented by an edit distance between phoneme strings of commands.

9. The information processing device of claim 3, wherein the score includes command brevity and the user's vocabulary knowledge.

10. The information processing device according to claim 9, wherein the conciseness of the command is expressed as the reciprocal of the number of moras of speech.

11. The information processing device according to claim 9, wherein the user's vocabulary knowledge is represented by a value based on the presence or absence of content words constituting the voice commands given by the user in the past.

12. The information processing device according to claim 2, wherein the command generation unit generates the command candidates using a language model.

13. The information processing device according to claim 1, wherein the function identification unit identifies the function by understanding the intent of the voice command using a language model.

14. The information processing device according to claim 13, wherein the function specification unit performs understanding of the intent of the voice command based on a voice recognition result of the voice command using a voice recognition model.

15. The information processing device according to claim 1, further comprising a voice synthesis unit that generates voice feedback to the user by voice synthesizing the alternative command.

16. The information processing device according to claim 1, wherein the function is any function executed by any device.

17. An information processing method in which an information processing device identifies a function based on understanding the voice intent of a voice command given by a user, and outputs an alternative command for executing the function in place of the voice command based on an evaluation of the voice command.

18. A program for causing a computer to execute a process of identifying a function based on understanding the voice intent of a voice command given by a user, and outputting an alternative command for executing the function in place of the voice command based on an evaluation of the voice command.

Images