Voice processing device, voice processing method and program

Abstract

To reduce misrecognition in voice recognition when voice information of multiple registered users is similar.SOLUTION: A voice processing device of one embodiment includes: a group information calculation unit that calculates group information for each user group where features of voice information are similar among users, from registration information where the voice information of each of multiple users is registered; a speech history coefficient calculation unit that calculates a speech history coefficient for each user based on a user's speech history; a selection unit that selects a predetermined number of users from among the users as recognition targets for the voice information based on the speech history coefficient; and an output unit that outputs recognition subject information for recognition subjects selected by the selection unit. The output unit, when the recognition subjects selected by the selection unit include multiple users belonging to the same user group, outputs the recognition subject information and information indicating that the multiple users belonging to the same user group are included.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to an audio processing device, an audio processing method, and a program. [Background technology]

[0002] Conventionally, there are processing devices that recognize spoken commands and execute processing. In the case of in-vehicle devices, if there is a passenger other than the user in the vehicle, words unintentionally spoken by the passenger are recognized as commands, so by registering the user's voice characteristics, speaker recognition of the voice information is also performed.

[0003] Patent Document 1 discloses a technique aimed at suppressing erroneous recognition in voice recognition caused by a microphone collecting not only the user's speech but also speeches of others and surrounding noise. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-86132 Summary of the Invention [Problem to be solved by the invention]

[0005] The present disclosure provides a speech processing device, a speech processing method, and a program that can reduce erroneous speech recognition when speech information of multiple registered users is similar. [Means for solving the problem]

[0006] The voice processing device according to the present disclosure includes a group information calculation unit that calculates group information for each user group in which the characteristics of the voice information of users are similar from registration information in which the voice information of each of multiple users is registered; an utterance history coefficient calculation unit that calculates an utterance history coefficient for each user based on the user's utterance history; a selection unit that selects a predetermined number of users from among the users based on the utterance history coefficients as targets for recognition of the voice information; and an output unit that outputs recognition target information of the targets for recognition selected by the selection unit, wherein the output unit outputs the recognition target information and information indicating that the targets for recognition selected by the selection unit include multiple users from the same user group. [Effects of the Invention]

[0007] According to the present disclosure, it is possible to reduce erroneous recognition of voice when the voice information of multiple registered users is similar. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of an application target of a voice processing device according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a configuration of hardware blocks of the voice processing device according to the embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of a functional block configuration of the voice processing device according to the embodiment. [Figure 4] FIG. 4 is a diagram showing an example of a registration information table generated by the recognition target person selection unit. [Figure 5] FIG. 5 is an explanatory diagram of the procedure for selecting a recognition target. [Figure 6] FIG. 6 is a diagram showing an output example of a recognition target person. [Figure 7] FIG. 7 is a diagram showing an example of an operation for manually correcting the selection of a recognition target person. [Figure 8]FIG. 8 is an explanatory diagram of Case 2 in which there are multiple members with the same group number among the selected recognition targets. [Figure 9] FIG. 9 is a diagram showing an example of an output of a UI screen in Case 2 and an example of an operation on the UI screen for correction. [Figure 10] FIG. 10 is a diagram showing an example of a flow for selecting a recognition target person. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of a voice processing device, a voice processing method, and a program according to the present disclosure will be described in detail with reference to the accompanying drawings.

[0010] (Embodiment) FIG. 1 is a diagram illustrating an example of an application target of a speech processing device according to an embodiment. First, the concept of speech recognition processing will be described. FIG. 1 shows a conceptual diagram of speech recognition processing. The speech recognition processing includes command recognition 1-1 that performs command recognition and speaker recognition 1-2 that performs speaker recognition.

[0011] Command recognition 1-1 determines whether an uttered word corresponds to a registered command, regardless of who the speaker is, and recognizes the command corresponding to the word.

[0012] Speaker Recognition 1-2 compares the voice information of the speaker who uttered the word with the registered voice information to recognize whether the speaker is a registered speaker. Here, "registered speaker" refers to a speaker whose voice information is registered.

[0013] Note that multiple pieces of voice information may be registered for each speaker. For example, if there are multiple types of commands, voice information corresponding to each command is registered for each speaker, and speaker recognition is performed based on the voice information corresponding to the command uttered by the speaker.

[0014] As shown by the arrows in Figure 1, words spoken by a speaker (Mr. A) are converted into voice signals via a microphone, for example, and voice recognition is performed based on the voice signals. In the voice recognition, if command recognition 1-1 is successful (OK) and speaker recognition 1-2 is successful (OK), the command successfully recognized by command recognition 1-1 is output, and the processing device executes that command.

[0015] Speaker recognition 1-2 acquires the speaker's voice information based on the voice signal, and performs speaker recognition using the voice information corresponding to the command recognized by command recognition 1-1, for example. Note that when speaker recognition 1-2 is turned off, it is possible to switch so that commands can be executed only by command recognition by command recognition 1-1.

[0016] (Description of the target of application of voice processing devices) Speech recognition begins after the voice information of the user who will use the voice recognition is registered in advance. If there are multiple users who will use the voice recognition, the voice information of each user is registered. Therefore, there will be multiple registered speakers whose voice information is registered.

[0017] Since there are multiple registered speakers, if there are multiple registered speakers with similar voice information among the registered speakers, there is a possibility that these users will mistakenly identify the actual speaker.

[0018] In this embodiment, the configuration of the speech processing device will be described in detail using an example in which it is combined with a device that performs speaker recognition using speech information from multiple registered speakers. The speech processing device may be used as a standalone device in combination with another speech recognition device, or may be pre-integrated into the speech recognition device, or the speech recognition device and speech processing device may be incorporated into an execution device that executes commands. The following description will be given using an example of an in-vehicle device such as a car navigation system as the execution device.

[0019] (Hardware configuration of audio processing device) 2 is a diagram illustrating an example of a hardware block configuration of a speech processing device according to an embodiment. As an example, the configuration of an in-vehicle device such as a car navigation system is illustrated. Note that the configuration of the in-vehicle device is merely an example and is not intended to be limiting.

[0020] 2 includes a CPU 21, a memory 22, a touch panel 23, a display 24, a storage device 25, a communication IF (interface) 26, a camera 27, a speaker 28, and a microphone 29. Each component is interconnected via a bus.

[0021] The CPU 21 is a central processing unit (CPU) that executes a predetermined program stored in the memory 22 to control each part and perform processing.

[0022] The memory 22 is a read-only memory (ROM) or a random access memory (RAM). The memory 22 stores predetermined programs and data. The memory 22 also has a work area that the CPU 21 uses for processing.

[0023] The touch panel 23 is a sensor that detects a touch position on the screen of the display 24.

[0024] The display 24 is a display such as a liquid crystal display.

[0025] The storage device 25 is a storage such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0026] The communication IF 26 is a communication interface that communicates with external devices. For example, the communication IF 26 connects to a predetermined network (such as the Internet) via wireless communication.

[0027] The camera 27 has an imaging device such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), and captures an image of an object to be imaged such as the interior of a vehicle.

[0028] The speaker 28 outputs predetermined sounds (such as operation sounds and notification sounds) and voices (such as response messages) that are reproduced by the CPU 21.

[0029] The microphone 29 converts the voice into an audio signal and inputs it.

[0030] 3 is a diagram illustrating an example of a functional block configuration of the speech processing device according to the embodiment. Each functional block of the speech processing device is realized by, for example, a CPU 21 executing a program in a memory 22. Note that only functions related to speech processing for speaker recognition are shown here, and other functions are not illustrated.

[0031] As shown in Fig. 3, the system includes a control unit 200, a speech input unit 201, a speaker registration / recognition selection unit 202, a speaker recognition unit 203, a user presentation unit 204, a user selection unit 205, a recognition target person selection unit 206, a feature calculation unit 207, a registered speaker database (registered speaker DB) 208, a similar speaker calculation unit 209, an auxiliary information acquisition unit 210, and an utterance history coefficient calculation unit 211. Here, the recognition target person selection unit 206 corresponds to the "selection unit". The user presentation unit 204 corresponds to the "output unit". The feature calculation unit 207 corresponds to the "speech information calculation unit". The similar speaker calculation unit 209 corresponds to the "group information calculation unit".

[0032] The control unit 200 comprehensively controls each of the functional blocks according to the operation mode of the audio processing device.

[0033] The audio input unit 201 receives an audio signal from the microphone 29 .

[0034] The speaker registration / recognition selection unit 202 receives a selection of either "speaker registration" or "speaker recognition" from the control unit 200.

[0035] The speaker recognition unit 203 performs predetermined processes related to speaker recognition, including speaker recognition based on a voice signal after speaker recognition has started.

[0036] The user presentation unit 204 receives output information from the speaker recognition unit 203 and presents the information to the user. For example, the user presentation unit 204 displays the information on the display 24. The user presentation unit 204 may present the information by audio output from the speaker 28. The presented information also includes information for advance preparation for speaker recognition.

[0037] The user selection unit 205 accepts a user selection for the information presented by the user presentation unit 204. For example, the user selection unit 205 accepts a selection operation by the user via the touch panel 23. The selection operation may be performed by an input means other than the touch panel 23, such as an input key. The user selection unit 205 may also accept a selection operation by voice by the user via command recognition.

[0038] In this embodiment, as an example, a configuration will be described in which information is presented on the display 24 and a selection operation by the user is accepted via the touch panel 23. However, other configurations may of course be used. For example, a configuration in which information is presented as audio output via the speaker 28 and the user makes a selection by voice via the microphone 29 may also be used.

[0039] The recognition target speaker selection unit 206 selects a predetermined number of registered speakers (users) as recognition targets from a list of registered speakers (users) whose speech information is registered, in order of the speech history coefficients, which are coefficients based on the speech history of each registered speaker.

[0040] Furthermore, if there is a possibility of misidentification in speaker recognition, for example, if there are registered speakers whose registered speech information is similar to each other among the selected recognition targets, the recognition target selection unit 206 will either exclude the similar registered speakers from the recognition targets using a predetermined method or add notification information to alert the user.

[0041] When the speaker registration / recognition selection unit 202 selects "speaker registration," the feature calculation unit 207 calculates speaker features from a speech signal, which is the speaker's speech input from the speech input unit 201, through a DNN, and registers the speaker features in the registered speaker DB 208. The speaker features are an example of "speech information," and are information indicating the characteristics of the speaker's speech.

[0042] The registered speaker DB 208 stores data on multiple registered speakers by associating registered speaker identification information, which is identification information of the registered speaker, with speaker features, which are the speaker's voice information. The registered speaker identification information is information that can uniquely identify the registered speaker, and an example is the name of the registered speaker (hereinafter, Mr. A, Mr. B, etc.). The stored database file corresponds to "registered information."

[0043] The similar speaker calculation unit 209 acquires registration information from the registered speaker DB 208 and calculates similar speaker groups. Speaker features may be acquired from the feature calculation unit 207 for each speaker registration. A similar speaker group is group information of registered speakers whose registered speaker features fall within a similarity range. The similarity range sets the range in which registered speakers may be mistaken for each other when speaker recognition is performed by the speaker recognition unit 203. As an example, the group information is calculated using the following method.

[0044] When registering each speaker, the similarity between each speaker is calculated for the feature vectors (d-vectors) of speaker features calculated based on the speaker's speech signal. Similarity is calculated using methods such as cosine similarity, which can determine similarity from the angle between two vectors. The cosine similarity of the feature vectors of the two people being compared is calculated for each registered speaker to determine the inter-speaker similarity. Then, if the inter-speaker similarity is equal to or exceeds a specified value that is considered to be within the similarity range, grouping is performed. A clustering method such as the k-means method may be used at this time. The grouped members are managed as group information for each group. This process is performed every time a new speaker is registered, and the group information is updated.

[0045] If there are multiple commands to be registered as speaker features, the similarity between each command between each speaker is calculated for the feature vector (d-vector) obtained for each command spoken by the speaker.The similarity calculated for each command is then averaged across commands to calculate the inter-speaker similarity.If the inter-speaker similarity is equal to or greater than a specified value, grouping is performed.

[0046] The auxiliary information acquisition unit 210 acquires information related to the frequency of utterances by registered speakers. As an example, the auxiliary information acquisition unit 210 acquires identification information of a registered speaker recognized as a speaker when speaker recognition is successful in the speaker recognition unit 203. In addition to the registered speaker identification information, the auxiliary information acquisition unit 210 may also acquire information such as the time and day of the week when speaker recognition is successful in the speaker recognition unit 203.

[0047] The speech history coefficient calculation unit 211 sets an utterance history for the registered speaker based on the information acquired by the auxiliary information acquisition unit 210, and calculates an utterance history coefficient for each registered speaker based on the set utterance history.

[0048] Here, an example of a method for calculating an utterance history coefficient based on the utterance history is shown. For example, if the number of registered speakers is N, the utterance history coefficient (C log(k)) is calculated for each registered speaker (k) using the following formula 1.

[0049]

number

[0050] Also, A frql : Weight for Freq_long_sr(k) A frqs : Weight for Freq_short_sr(k) A wd : Weight for W_day(k) Then, each weight (A frql , A frqs , A wd ) may be a fixed value set by the user or may be variable by the system.

[0051] Freq_long_sr(k) is a formula for calculating the speech rate of a recognized speaker (k). For example, the speech rate is calculated by dividing the number of times each speaker is recognized from the speaker history by the sum of the number of times all speakers are recognized.

[0052] Freq_short_sr(k) is a formula for calculating the proportion of utterances by a speaker whose utterances were recognized within an even shorter period of time. The number of times each speaker was recognized may be weighted so that the value increases in ascending order of the number of times each speaker was recognized within the current time. This allows the speech history coefficient of a speaker whose utterance was recognized within a period closer to the current time to be increased.

[0053] W_day(k) is a formula for calculating the speech rate based on the same day of the week or the same time period each week. For each speaker, the speech history for each day of the week or time period is used to quantify the speech rate based on the deviation of the current day of the week or time period from the respective standard.

[0054] For example, the frequency of use on what day and time is calculated from the time records included in the speech history, and the degree of deviation in time from the actual day of use and time period is quantified. The smaller the deviation in time from the standard, the larger the value is set. Therefore, the more a user uses the device on the same date, day of the week, or time period each time, the larger the speech history coefficient for that user's use on the same date, day of the week, or time period will be, and the accuracy of selecting users who use the device on the same date, day of the week, or time period will be improved.

[0055] In the case of a navigation system, W_day(k) may be calculated by linking the frequency of setting the same destination in the same time period on the same day of the week and voice operations to the speech history.

[0056] Each part of the functional block shown in FIG. 3 performs the following operation depending on the operation mode of the speech processing device (for example, a speaker registration mode or a speaker recognition mode).

[0057] (Speaker registration mode) When the user selection unit 205 of the speech processing device accepts an operation for speaker registration, the control unit 200 switches the selection of the speaker registration / recognition selection unit 202 to "speaker registration" and the speech processing device enters the speaker registration mode.

[0058] In the speaker registration mode, the speaker registration / recognition selection unit 202 receives a speaker's voice signal from the voice input unit 201 and outputs it to the feature calculation unit 207. The feature calculation unit 207 calculates speaker features based on the voice signal and outputs them to the registration speaker DB 208. The registration speaker DB 208 registers the speaker features output from the feature calculation unit 207 in association with the registration speaker identification information of the speaker. In the case of an unregistered speaker, the registration speaker DB 208 registers the speaker features in association with new registration speaker identification information.

[0059] Furthermore, when the feature calculation unit 207 registers speaker features in the registered speaker DB 208, it notifies the similar speaker calculation unit 209 that the registration has been made in the registered speaker DB 208. Based on this notification, the similar speaker calculation unit 209 detects that the speaker features in the registered speaker DB 208 have been updated, and each time a speaker feature is registered, it obtains the registration information in the registered speaker DB 208 and calculates a similar speaker group.

[0060] (Speaker recognition mode) In the voice processing device, the control unit 200 switches the selection of the speaker registration / recognition selection unit 202 to "speaker recognition" when the power is normally turned on or after the speaker registration mode ends, and the mode is switched to the speaker recognition mode.

[0061] In the speaker recognition mode, the control unit 200 turns off the input of the voice signal to the speaker recognition unit 203 until advance preparation for speaker recognition is completed.

[0062] The similar speaker calculation unit 209 outputs the latest registration information of the registered speaker DB 208 and information indicating the similar speaker group calculated from the latest registration information to the recognition candidate selection unit 206.

[0063] The utterance history coefficient calculation unit 211 updates the utterance history based on the information acquired by the auxiliary information acquisition unit 210, and calculates an utterance history coefficient for each registered speaker based on the updated utterance history.

[0064] The recognition candidate selection unit 206 acquires the utterance history coefficients calculated by the utterance history coefficient calculation unit 211, and selects a predetermined number of registered speakers as recognition candidates from the registered speakers (list) included in the registration information output by the similar speaker calculation unit 209 in an order based on the utterance history coefficients. Furthermore, if there are registered speakers whose registered speech information is similar to each other among the selected recognition candidates, the recognition candidate selection unit 206 either excludes the similar registered speakers from the recognition candidates using a predetermined method based on the information of the similar speaker group, or adds notification information to alert the user. The selection of recognition candidates will be described in detail later using figures.

[0065] The recognition target person selection unit 206 outputs the registration information and notification information of the selected recognition target person to the speaker recognition unit 203.

[0066] The speaker recognition unit 203 outputs confirmation information of the registered speaker who has become the recognition target to the user presentation unit 204 based on the registration information of the recognition target and the notification information output from the recognition target selection unit 206 .

[0067] Thereafter, upon receiving a predetermined operation from the user selection unit 205 , the recognition target person selection unit 206 outputs output information corresponding to the predetermined operation to the speaker recognition unit 203 .

[0068] For example, when the recognition target selecting unit 206 receives an operation to delete a specific registered speaker from the registered speakers selected as recognition targets, it deletes the registered speaker from the recognition targets and outputs registration information in which the deleted number of speakers are added to the recognition targets in the order based on the utterance history coefficients to the speaker recognition unit 203. The output information of the user presentation unit 204 is updated based on this output information.

[0069] Furthermore, when the recognition target selecting unit 206 receives an operation to add a registered speaker by later selection from among the registered speakers selected as recognition targets, the unit 206 adds the registered speaker to be added, including the number of deleted speakers.

[0070] The selection of the registered speaker to be added can be performed by the recognition target selecting unit 206 outputting a list of registered speakers included in the entire registration information before the selection of the recognition target to the speaker recognition unit 203, and the speaker recognition unit 203 outputting the list of registered speakers to the user presenting unit 204. For example, the recognition target selecting unit 206 may receive an operation from the user selecting unit 205 to display a list of registered speakers included in the registration information, and output the list of registered speakers to the speaker recognition unit 203.

[0071] Once the advance preparation is complete, the control unit 200 turns on the input of the voice signal to the speaker recognition unit 203. The speaker recognition unit 203 calculates speaker features based on the input voice signal, compares the calculated speaker features with the speaker features of each registered speaker to be recognized that was determined in the advance preparation, and recognizes as the speaker any registered speaker whose similarity with the speaker features of the input voice signal exceeds a threshold.

[0072] The in-vehicle device 2 executes the command of the voice signal of the speaker recognized by the command recognition 1-1 based on the result of speaker recognition by the speaker recognition unit 203 (speaker recognition 1-2).

[0073] (Description of Recognition Target Person Selection by Recognition Target Person Selection Unit 206) Next, the selection of a recognition target person by the recognition target person selection unit 206 will be described in detail with reference to Figs. 4 and 5. Fig. 4 is a diagram showing an example of a registration information table generated by the recognition target person selection unit 206. The registration information table 300 shown in Fig. 4 is obtained by adding a column for a similar speaker group and a column for an utterance history coefficient to the acquired registration information. Fig. 4 shows a list of information on 20 registered speakers. Note that the number of registered speakers is an example and is not limited to this.

[0074] The registration information table 300 shown in Figure 4 is a table having a "Speaker No." column 301, a "Registered Speaker Name" column 302, a "Speech History Coefficient" column 303, a "Feature Similarity Group No." column 304, a "Command 1" column 305, a "Command 2" column 306, ..., and a "Command M" column 307.

[0075] A serial number is set in the "Speaker No." column 301. The name of the registered speaker is set in the "Registered Speaker Name" column 302 in association with the number in the "Speaker No." column 301. Here, the name of the registered speaker is an example of registered speaker identification information. When presented to the user, this name is presented.

[0076] The "Speech History Coefficient" column 303 sets the utterance history coefficient of each registered speaker. Each utterance history coefficient is a value calculated by the utterance history coefficient calculation unit 211. It can be inferred that a registered speaker with a higher utterance history coefficient value uses speaker recognition more frequently, that is, the user is more likely to use speaker recognition this time as well. Conversely, it can be inferred that a registered speaker with a low utterance history coefficient value, especially a registered speaker with a value below a predetermined value, is a user who is more likely to not use speaker recognition this time as well.

[0077] In the "Similar Feature Group No." column 304, a group number is set to distinguish each similar feature group. The same group number is set for registered speakers in the same group based on information calculated by the similar speaker calculation unit 209 as registered speakers in the same group. In the example shown in FIG. 4, Mr. A and Mr. B are in the same group 401, and group number 1 is set. Mr. F, Mr. G, and Mr. J are in the same group 402, and group number 2 is set. Mr. P and Mr. T are in the same group 403, and group number 3 is set.

[0078] Each piece of data in the "Command 1" column 305, "Command 2" column 306, ... "Command M" column 307 is data on the speech information (speaker features) of each registered speaker corresponding to the multiple registered commands (Command 1, Command 2, ... Command M).

[0079] 5 is a diagram illustrating the procedure for selecting a recognition target person. Since this is an explanatory diagram, the command column in the registration information table 300 is omitted from the illustration.

[0080] 5 shows the arrangement of data before and after sorting the registration information table 300 by the values ​​in the "utterance history coefficient" column 303. As indicated by the arrow, after sorting, the data is arranged as shown on the right.

[0081] After sorting, the words are arranged in descending order of the value in the "utterance history coefficient" column 303, as shown in FIG.

[0082] Recognition targets are selected in descending order of the value in the "utterance history coefficient" column 303. The number of recognition targets may be determined arbitrarily by the user, or may be set automatically depending on the situation in which the system is used. For example, when used in a vehicle, the maximum number of passengers is determined depending on the vehicle model, and the number of recognition targets may be set automatically accordingly. The number of recognition targets may be set manually by the user, or may be set automatically. Furthermore, the automatically set number may be manually reset by the user.

[0083] If the number is to be set automatically, for example, a vehicle is equipped with a seat belt fastening sensor or a seat sensor, and the outputs of these sensors can be combined to detect the number of occupants, and the detected number can be automatically set as the number of people to be recognized.

[0084] In this example, the condition for the number of recognition targets is set to "6". In addition, a condition for the lower limit of the speech history coefficient, "15", is set. First, by selecting people in descending order of the value in the "Speech History Coefficient" column 303, six people are determined: E, L, K, A, P, and B. However, of these six people, A and B are members with the same group number in the "Similar Feature Group No." column 304. In this case, there is a high possibility of erroneous recognition when speaker recognition is performed, so a condition for excluding members below the lower limit of the speech history coefficient, "15", is used to exclude members with the same group number as much as possible.

[0085] Comparing the "utterance history coefficient" column 303 of A and B, A's value is higher, so A is retained. As for B, B's value is compared with the lower limit "15", and B's value is below the lower limit "15", meaning that it is highly likely that speaker recognition will not be used this time either, so B is deleted. Having deleted B, F, who has the next highest value in the "utterance history coefficient" column 303, is selected. Using this method, the six people in the selection frame 500 become recognition candidates.

[0086] Next, examples of output of six people who are the recognition targets by the user presentation unit 204 will be described in detail with reference to FIGS.

[0087] Fig. 6 is a diagram showing an example of output of recognition target persons. Fig. 6 shows an example of presenting information by the in-vehicle device 2. As shown in Fig. 6, the in-vehicle device 2 provides voice guidance through the speaker 28. In addition, a list of six recognition target persons is output to the display 24 of the in-vehicle device 2, and user operation is accepted.

[0088] The display 24 displays a UI (User Interface) screen 610. The UI screen 610 is provided with an avatar display section 617, a guidance information display section 611, a list display section 612, a speaker recognition OFF button 613, a speaker selection button 614, a timeout time display section 616, and a map screen button 615.

[0089] The avatar display unit 617 performs a text readout operation to encourage the user to view the content displayed in the guidance information display unit 611. The guidance information display unit 611 displays guidance such as the number of recognition targets in text. The list display unit 612 displays a list of registered speaker names (registered speaker identification information) of the six selected recognition targets. The list of registered speaker names of the six recognition targets is an example of "recognition target information." The timeout time display unit 616 displays the timeout time for checking the list.

[0090] The speaker recognition OFF button 613, speaker selection button 614, and map screen button 615 are each a user operation button. The speaker recognition OFF button 613 is an operation button for switching speaker recognition ON and OFF. The speaker selection button 614 is an operation button for switching to a speaker selection screen for manually correcting the selection of a recognition target. The map screen button 615 is an operation button for switching to a map screen of the navigation of the in-vehicle device 2.

[0091] The operation keys arranged below the screen of the display 24 are dedicated to operating in-vehicle devices such as navigation systems.

[0092] 7 is a diagram showing an example of an operation for manually correcting the selection of a recognition target. By touching the speaker selection button 614 on the UI screen 610, the UI screen switches to a correction UI screen 620, as indicated by the arrow in FIG.

[0093] The UI screen 620 for correction is provided with a correction guidance information display section 621, a list display section 622 for correction, a speaker registration operation button 623, and a map screen button 615.

[0094] The correction guidance information display unit 621 displays textual guidance for correction. The correction list display unit 622 acquires a list of all registered speakers and displays it, allowing selection of addition or deletion for each registered speaker. The speaker registration operation button 623 is an operation button for instructing a change to speaker registration mode. The speaker registration operation button 623 is used to register voice information of a new user or add voice information of a user.

[0095] Here, a specific example will be given to explain in detail the correction list display section 622. As an example, as shown in Fig. 7, the correction list display section 622 displays a registered speaker name column 701, a selection display column 702, and a similar speaker G (similar speaker group) column 703 in association with each other, and is capable of scrolling so that all registered speakers can be corrected.

[0096] The registered speaker name field 701 displays a list of registered speakers. The selection display field 702 is a field for directly selecting or deselecting a registered speaker. In this example, check boxes are provided, and the selection or deselection of each registered speaker is directly operated by clicking the check boxes.

[0097] (Case 1) Consider a case where the automatic selection output result shown in the list display section 612 of the UI screen 610 in FIG. 7 is actually used by Mr. I. In this case, Mr. I is not included in the automatically selected recognition candidates. Mr. I can confirm from the list display section 612 of the UI screen 610 that he or she is not included, and therefore touches the speaker selection button 614 to switch to the correction UI screen 620. In the correction list display section 622, Mr. I moves the cursor 624 to the check box 625 corresponding to Mr. I by touching or other operation and turns on the check box 625. The user selection unit 205 notifies the recognition candidate selection unit 206 that Mr. I has been added as a recognition candidate, and Mr. I is added as a recognition candidate. Furthermore, upon addition, the person with the lowest value in the “utterance history coefficient” field 303 may be excluded from the recognition candidates. When the UI screen 610 is returned, the list of registered speaker names in the list display section 612 is updated. Corrections in the correction list display section 622 can be made repeatedly, allowing correction even if an actual vehicle passenger is not automatically selected.

[0098] (Case 2) Fig. 8 is an explanatory diagram of Case 2 in which there are multiple members with the same group number among the selected recognition targets. As shown in Fig. 8, if the speech history coefficients of Mr. A and Mr. B, who have the same group number, are both high in the registration information table 300 before sorting, after sorting, Mr. A and Mr. B are included in the top two people and their speech history coefficients are equal to or greater than the lower limit value "15", and therefore they are included in the six recognition targets 501.

[0099] In this case, too, erroneous recognition can be suppressed by selecting the actual user between A and B and deleting the other.

[0100] 9 is a diagram showing an example of output from the UI screen 610 in Case 2 and an example of operation on the correction UI screen 620. As shown in FIG. 9, the list of recognition targets on the UI screen 610 includes multiple members with the same group number. Because there is a high possibility of misrecognition between members with the same group number, notification information calling the user's attention is included on the screen. As an example, in FIG. 9, the guidance information display unit 611 displays the attention content in text. Also, in the list display unit 612, a symbol indicating an attention is displayed next to the name of the member with the same group number, such as the group number "1" of similar speakers followed by an exclamation mark ("!") in red.

[0101] By operating the correction UI screen 620, it is possible to remove either person A or B from the list of people to be recognized by unchecking the checkbox for either person A or B.

[0102] It should be noted that when the number of members in the similar speaker group becomes one by operating the correction UI screen 620, the warning notation (text or symbols) may disappear.

[0103] (Selection flow of the recognition target) 10 is a diagram showing an example of a selection flow of a recognition target person. This selection flow is performed as a pre-processing for the speech recognition mode immediately after the speech processing device is powered on.

[0104] First, the speech processing device calculates the utterance history coefficients of registered speakers from their utterance histories, and narrows down the candidates for recognition based on the calculated utterance history coefficients (step S1). The candidates for recognition are narrowed down in descending order of the registered speakers with the highest utterance history coefficients, until the number of candidates for recognition reaches a predetermined number (the number set, etc.).

[0105] Next, the speech processing device determines whether the narrowed-down recognition targets include multiple members of the same group (step S2).

[0106] In step S2, if the narrowed down recognition targets do not include multiple members of the same group (step S2: No), the speech processing device proceeds to step S5.

[0107] On the other hand, in step S2, if the narrowed-down recognition targets include multiple members of the same group (step S2: Yes), the speech processing device determines whether the speech history coefficients of the members of the same group are equal to or greater than a lower limit (step S3).

[0108] Next, in step S3, if there is a member in the same group among the narrowed-down recognition targets whose speech history coefficient is below the lower limit value (step S3: No), the speech processing device excludes from the recognition targets all members other than those above the lower limit value or all members other than the initially selected one, and selects a person outside the same group with the next highest speech history coefficient (step S4), and proceeds to step S5.

[0109] On the other hand, in step S3, if the utterance history coefficients of the members in the same group are equal to or greater than the lower limit (step S3: Yes), the speech processing device displays a list of recognition targets to the user (step S5).

[0110] Next, the voice processing device determines whether "speaker selection" (speaker selection button 614) has been selected (step S6).

[0111] In step S6, if "speaker selection" is not selected (step S6: No), the speech processing device proceeds to step S8.

[0112] On the other hand, in step S6, if "speaker selection" is selected (step S6: Yes), the speech processing device receives a selection operation of a recognition target by the user (step S7). The selection operation of a recognition target by the user corresponds to an addition or deletion operation on the correction UI screen 620. When an addition or deletion operation is performed, the speech processing device adds or deletes the recognition target.

[0113] Next, the voice processing device determines whether "speaker registration" (speaker registration operation button 623) has been selected (step S8).

[0114] In step S8, if "speaker registration" is not selected (step S8: No), the voice processing device transitions to the normal navigation operation screen, and starts speaker recognition for the recognition target person selected in the processing of steps S1 to S7.

[0115] On the other hand, in step S8, if "speaker registration" is selected (step S8: Yes), the speech processing device switches to a speaker registration mode and accepts speaker registration on the speaker registration screen. After speaker registration, the speech processing device switches to a speaker recognition mode and performs speaker recognition target selection (processing included in steps S1 to S7) and speaker recognition.

[0116] (Variation 1) In this embodiment, as an example of a case where a recognition target includes multiple members of the same similar speaker group, a case where both A and B, who are members of group number 1, have high speech history coefficients that exceed the lower limit (see FIG. 8) has been described. In this example case, it has been explained that the user can decide whether to confirm the recognition target while including both A and B, or to exclude one of them.

[0117] Therefore, as a modified example of the embodiment, a condition may be added as a condition when the recognition-target person selection unit 206 automatically selects a recognition target person, whereby the person with a lower speaker history coefficient is automatically excluded when there is a difference of a predetermined value or more in the speaker history coefficients. When this condition is added, the recognition-target person selection unit 206 can exclude the person with a lower speaker history coefficient in advance if the difference in the speaker history coefficients of Mr. A and Mr. B is a predetermined value or more. In this case, the user operation to exclude the person with a lower speaker history coefficient after the recognition targets are presented is not required.

[0118] (Variation 2) In the embodiment, it has been described that the number of people to be recognized can be narrowed down using sensors. As an example, in a vehicle, a seat belt fastening sensor or a seat sensor is provided, and the number of occupants can be detected by combining the outputs of these sensors. In addition, it is also possible to detect the number of people using an image sensor such as a camera.

[0119] In this way, by combining sensor detection to narrow down the number of people to be recognized, if there is only one actual passenger, it becomes possible to detect that as one person. In this case, regardless of the lower limit condition or the additional condition set in Modification 1, the person to be recognized can be automatically narrowed down to Person A, who is the speaker with the highest utterance history coefficient, and Person B can be automatically excluded.

[0120] (Variation 3) In the calculation formula 1 for calculating the utterance history coefficient described in the embodiment, the weight of each intermediate parameter may be changed with use. For example, in the initial stage of use, the number of voice operations is small, so the calculation result of the utterance frequency of users using the same day of the week or the same time period is low in reliability, but as use progresses, the matching frequency of users using the same day of the week or the same time period gradually increases and the reliability becomes higher. Therefore, the weighting coefficient Awd of W_day(k), which is calculated based on the same day of the week or the same time period, may be gradually increased with use.

[0121] (Variation 4) In the embodiment, the registered speaker DB 208 registered by the speech processing device may be provided in an external storage such as a cloud storage, etc. The speech history coefficient calculation unit 211 may acquire some or all of the information collected in the external storage and set the speaker history.

[0122] As described above, the speech processing device of this embodiment or the modified example performs processing to limit the number of recognition targets for speaker recognition to a predetermined number from among the registered speakers registered in the registration information. The recognition targets are determined based on the speech history coefficient calculated from the speech history of each registered speaker and the information of similar speaker groups calculated from the registered speech information. As a result, recognition targets that are more likely to be used are selected preferentially, and since it is clear that registered speakers from similar speaker groups are included, misrecognition can be reduced, thereby improving usability for the user.

[0123] Furthermore, even if the recognition target persons selected by the speech processing device include registered speakers in a similar speaker group or do not include the actual user, the recognition target persons selected by the speech processing device are presented to the user, and the user can also operate to correct the recognition target persons.

[0124] By reflecting this correction in the speech history, the accuracy of the speech history coefficients calculated by the speech processing device based on the speech history increases, and the accuracy of the speech processing device's selection of recognition targets improves as the device continues to be used repeatedly.

[0125] The present disclosure can be realized in software, hardware, or software in conjunction with hardware.

[0126] The present disclosure may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. A program product is a computer-readable medium on which a computer program is recorded.

[0127] In addition, a program recording some or all of the procedures may be provided by recording it on a recording medium, or stored in ROM and provided as a computer-configured information processing device, or the program may be downloaded via a network and executed by a computer. The CPU of the computer performs processing by reading and executing the program.

[0128] Although the embodiments have been described above with reference to the drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications or alterations within the scope of the claims. Such modifications or alterations are also considered to fall within the technical scope of the present disclosure. Furthermore, the components in the embodiments may be combined in any manner without departing from the spirit of the present disclosure.

[0129] (Addendum) Aspects of the present disclosure are, for example, as follows. (Item 1) a group information calculation unit that calculates group information of a user group in which the voice information of each of a plurality of users is similar in characteristics to each other from registered information in which the voice information of each of the users is registered; an utterance history coefficient calculation unit that calculates an utterance history coefficient for each user based on the utterance history of the user; a selection unit that selects a predetermined number of users from among the users based on the speech history coefficient as targets for recognition of the voice information; an output unit that outputs target person information of the target selected by the selection unit; and The output unit When the recognition targets selected by the selection unit include a plurality of users belonging to the same user group, output the recognition target information and information indicating that the recognition targets include a plurality of users belonging to the same user group. Audio processing device. (Item 2) the utterance history coefficient calculation unit calculates the utterance history coefficient so that a value of the utterance history coefficient is higher for a user corresponding to voice information that has been recognized as a speech more frequently in the past among the voice information of the registered users, Item 1. A voice processing device according to item 1. (Item 3) the utterance history coefficient calculation unit calculates the utterance history coefficient so that the value of the utterance history coefficient increases as the number of times that the utterance history coefficient has been recognized as an utterance in the past within a predetermined period increases among the registered voice information of the user. Item 3. A voice processing device according to item 1 or 2. (Item 4) the utterance history coefficient calculation unit calculates the utterance history coefficient so that the value of the utterance history coefficient is higher for a user corresponding to voice information of the registered user whose voice information is closer to a day of the week or a time period in which an utterance has been recognized many times in the past. 4. A voice processing device according to any one of items 1 to 3. (Item 5) When the recognition targets to be selected include a plurality of users belonging to the same user group, the selection unit deletes from the recognition targets, among the plurality of users belonging to the same user group, users whose utterance history coefficients satisfy a predetermined condition based on the utterance history coefficients of the plurality of users belonging to the same user group. 5. A voice processing device according to any one of items 1 to 4. (Item 6) a user selection unit configured to receive from a user a selection of adding or deleting a recognition target person to or from the recognition target person information output by the output unit; 6. A voice processing device according to any one of items 1 to 5. (Item 7) a voice input unit for inputting a speaker's voice; a speaker recognition unit that recognizes a speaker of the voice input from the voice input unit based on the voice information corresponding to the recognition target speaker of the recognition target; 7. The audio processing device according to claim 1, further comprising: (Item 8) a voice input unit for inputting a speaker's voice; a speaker recognition unit that recognizes a speaker of the voice input from the voice input unit based on the voice information corresponding to the recognition target speaker of the recognition target; a voice information calculation unit that calculates the voice information based on a voice signal input from the voice input unit and registers the voice information in the registration information; 7. The audio processing device according to any one of items 1 to 6, further comprising: (Item 9) calculating group information for each user group in which the voice information of each of a plurality of users is similar in characteristics to each other from registered information in which the voice information of each of the plurality of users is registered; calculating an utterance history coefficient for each user based on the utterance history of the user; selecting a predetermined number of users from among the users based on the speech history coefficient as targets for recognizing the voice information; outputting information about the person to be recognized that has been selected; a step of outputting the recognition target information and information indicating that the plurality of users belonging to the same user group are included in the selected recognition targets when the plurality of users belonging to the same user group are included in the selected recognition targets; An audio processing method comprising: (Item 10) On the computer, a group information calculation unit that calculates group information for each user group in which users have similar characteristics of the voice information from registration information in which each of the voice information of a plurality of users is registered; an utterance history coefficient calculation unit that calculates an utterance history coefficient for each user based on the utterance history of the user; a selection unit that selects a predetermined number of users from among the users based on the speech history coefficient as targets for recognition of the voice information; an output unit that outputs target person information of the target selected by the selection unit; It functions as When the recognition targets selected by the selection unit include a plurality of users belonging to the same user group, outputting the recognition target information and information indicating that the recognition targets include a plurality of users belonging to the same user group to the output unit. program. [Explanation of symbols]

[0130] 1-1 Command recognition 1-2 Speaker Recognition 2. In-vehicle equipment (audio processing device) 200 control section 201 Audio input section 202 Speaker registration / recognition selection unit 203 Speaker Recognition Unit 204 User Presentation Unit 205 User Selection Section 206 Recognition Target Selection Unit 207 Feature Calculation Unit 208 Registered Speaker Database 209 Similar Speaker Calculation Unit 210 Auxiliary Information Acquisition Department 211 Speech history coefficient calculation unit

Claims

1. a group information calculation unit that calculates group information of a user group in which the voice information of each of a plurality of users is similar in characteristics to each other from registered information in which the voice information of each of the users is registered; an utterance history coefficient calculation unit that calculates an utterance history coefficient for each user based on the utterance history of the user; a selection unit that selects a predetermined number of users from among the users based on the speech history coefficient as targets for recognition of the voice information; an output unit that outputs target person information of the target selected by the selection unit; and The output unit When the recognition targets selected by the selection unit include a plurality of users belonging to the same user group, output the recognition target information and information indicating that the recognition targets include a plurality of users belonging to the same user group. Audio processing device.

2. the utterance history coefficient calculation unit calculates the utterance history coefficient so that a value of the utterance history coefficient is higher for a user corresponding to voice information that has been recognized as a speech more frequently in the past among the voice information of the registered users, The audio processing device according to claim 1 .

3. the utterance history coefficient calculation unit calculates the utterance history coefficient so that the value of the utterance history coefficient increases as the number of times that the utterance history coefficient has been recognized as an utterance in the past within a predetermined period increases among the registered voice information of the user. The audio processing device according to claim 1 .

4. the utterance history coefficient calculation unit calculates the utterance history coefficient so that the value of the utterance history coefficient is higher for a user corresponding to voice information of the registered user whose voice information is closer to a day of the week or a time period in which an utterance has been recognized many times in the past. The audio processing device according to claim 1 .

5. When the recognition targets to be selected include a plurality of users belonging to the same user group, the selection unit deletes from the recognition targets, among the plurality of users belonging to the same user group, users whose utterance history coefficients satisfy a predetermined condition based on the utterance history coefficients of the plurality of users belonging to the same user group.

5. The audio processing device according to claim 1.

6. a user selection unit configured to receive from a user a selection of adding or deleting a recognition target person to or from the recognition target person information output by the output unit; 5. The audio processing device according to claim 1.

7. a voice input unit for inputting a speaker's voice; a speaker recognition unit that recognizes a speaker of the voice input from the voice input unit based on the voice information corresponding to the recognition target speaker of the recognition target; 5. The audio processing device according to claim 1, further comprising:

8. The audio processing device according to claim 7 , further comprising an audio information calculation unit that calculates the audio information based on the audio signal input from the audio input unit and registers the calculated audio information in the registration information.

9. calculating group information for each user group in which the voice information of each of a plurality of users is similar in characteristics to each other from registered information in which the voice information of each of the plurality of users is registered; calculating an utterance history coefficient for each user based on the utterance history of the user; selecting a predetermined number of users from among the users based on the speech history coefficient as targets for recognizing the voice information; outputting information about the person to be recognized that has been selected; a step of outputting the recognition target information and information indicating that the plurality of users belonging to the same user group are included in the selected recognition targets when the plurality of users belonging to the same user group are included in the selected recognition targets; An audio processing method comprising:

10. On the computer, a group information calculation unit that calculates group information for each user group in which users have similar characteristics of the voice information from registration information in which each of the voice information of a plurality of users is registered; an utterance history coefficient calculation unit that calculates an utterance history coefficient for each user based on the utterance history of the user; a selection unit that selects a predetermined number of users from among the users based on the speech history coefficient as targets for recognition of the voice information; an output unit that outputs target person information of the target selected by the selection unit; It functions as When the recognition targets selected by the selection unit include a plurality of users belonging to the same user group, outputting the recognition target information and information indicating that the recognition targets include a plurality of users belonging to the same user group to the output unit. program.

Images