Voice registration device and voice registration method

Abstract

A voice registration device comprising: an acquisition unit that acquires an audio signal of an uttered voice of a speaker and speaker information that can identify the speaker; a registration unit that associates the acquired audio signal with the acquired speaker information and registers the same in a database; a progress determination unit that repeatedly determines the registration progress of the audio signal to be registered in the database with respect to the registration target amount of the audio signal to be registered in the database; and a notification unit that notifies the determined registration progress.

Description

[Technical Field]

[0001] The present disclosure relates to a voice registration device and a voice registration method. [Background technology]

[0002] Patent Document 1 discloses a telephone device that, when it is determined that the total acquisition time length of at least one or more voiceprint data corresponding to a telephone number in a telephone directory and the same telephone number as the acquired telephone number is longer than the time required for voiceprint matching, associates the acquired telephone number with the voiceprint data and stores them. The telephone device acquires a received voice, extracts voiceprint data from the acquired received voice, measures the acquisition time of the received voice, and, if the acquired telephone number is in the telephone directory, associates the acquired telephone number with the extracted voiceprint data and stores them. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2016-53598 Summary of the Invention [Problem to be solved by the invention]

[0004] Patent Document 1 discloses a communication device that, when it is determined that the total acquisition time length of at least one or more voiceprint data corresponding to a telephone number in a telephone directory and the same telephone number as the acquired telephone number is longer than the time required for voiceprint matching, associates the acquired telephone number with the voiceprint data and stores the associated telephone number. However, the above-mentioned communication device collects the spoken voice (voiceprint data) based on the speaker's own operation, and when it is determined that the total acquisition time length of the voiceprint data is longer than the time required for voiceprint matching, it prompts the speaker to choose whether or not to register the voiceprint data. Therefore, when the total acquisition time length of the voiceprint data is shorter than the time required for voiceprint matching, the communication device has a problem in that it cannot store new voiceprint data if the speaker chooses not to register the voiceprint data.

[0005] Another method for registering voiceprint data is to extract only the speaker's voice from the voice of the conversation between the speaker and the operator, and store the extracted speaker's voice as voiceprint data. In this method of registering voiceprint data, the operator cannot know in real time the registration status of the speaker's voiceprint data that is currently acquired and stored, which can lead to a delay in responding to a customer during a call, or an inability to acquire and register sufficient voiceprint data during a call.

[0006] The present disclosure has been devised in consideration of the above-described conventional situation, and aims to provide a speech voice registration device and a speech voice registration method that notify the current registration status of speech voice signals in real time and assist operators in the registration work of speech voice signals. [Means for solving the problem]

[0007] The present disclosure provides an acquisition unit that acquires a speech signal of a speaker's speech and speaker information that can identify the speaker; a registration unit that associates the acquired speech signal with the speaker information and registers them in a database; and a registration unit that calculates a registration progress of speech signals to be registered in the database, a signal length of the registered speech signals, and a number of sound types included in the registered speech signals, relative to a registration target amount of speech signals to be registered in the database. and the number of sound types for each sound a selection unit that selects a plurality of questions to the speaker based on the registration progress, the target number of sound types to be registered, and the number of sound types included in the voice signal to be registered; and a notification unit that generates and outputs a progress notification screen that notifies the speaker of the registration progress, wherein the selection unit determines a priority order for each of the selected questions, and the notification unit notifies the speaker of each of the selected questions, the priority order of the question, the registration progress, and the number of sound types included in the audio signal and the number of sound types for each sound; The present invention provides a voice registration device that generates and outputs the progress notification screen including the above.

[0008] The present disclosure also provides a voice registration method executed by one or more computers, which acquires a voice signal of a speaker's speech and speaker information capable of identifying the speaker, associates the acquired voice signal with the speaker information and registers them in a database, and calculates a registration progress of the voice signal to be registered in the database and a signal length of the registered voice signal with respect to a registration target amount of the voice signal to be registered in the database. , in the database Included in the audio signal to be registered Number of sounds and each sound repeatedly determining the number of sound types, and selecting a plurality of questions to the speaker based on the degree of registration progress, the target number of sound types to be registered, and the number of sound types included in the voice signal to be registered; determining a priority order for each of the selected questions; and determining a priority order for each of the selected questions, the priority order for each of the questions, and the degree of registration progress. the number of sound types included in the audio signal and the number of sound types for each sound; and generating and outputting a progress notification screen that notifies the user of the progress of the voice registration. [Effects of the Invention]

[0009] According to the present disclosure, the current registration status of the speech voice signal is notified in real time, and the registration work of the speech voice signal performed by the operator is supported. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing an example of a use case of the voice authentication system according to the first embodiment. [Figure 2] FIG. 1 is a block diagram showing an example of the internal configuration of an authentication analysis device according to a first embodiment. [Figure 3] FIG. 1 is a diagram illustrating an example of emotion identification processing in the first embodiment. [Figure 4] FIG. 1 is a diagram illustrating an example of registering a speech signal. [Figure 5] A diagram showing an example of a conversation between an operator and a user [Figure 6] FIG. 10 is a diagram illustrating an example of speech speed conversion processing. [Figure 7] Flowchart for explaining an example of voice registration processing in the first embodiment [Figure 8] Flowchart for explaining an example of voice registration processing in the first embodiment [Figure 9] A diagram illustrating an example of a progress notification screen for a new user. [Figure 10] A diagram illustrating an example of a progress notification screen for registered users. [Figure 11] A diagram illustrating an example of a progress notification screen for registered users. [Figure 12] FIG. 10 is a block diagram showing an example of the internal configuration of an authentication analysis device according to a second embodiment. [Figure 13] FIG. 10 is a diagram illustrating an example of speaker change identification processing. [Figure 14] FIG. 10 is a diagram illustrating an example of emotion identification processing in the second embodiment. [Figure 15] Flowchart for explaining an example of voice registration processing in the second embodiment [Figure 16] Flowchart for explaining an example of voice registration processing in the second embodiment [Figure 17] FIG. 10 is a block diagram showing an example of the internal configuration of an authentication analysis device according to a third embodiment. [Figure 18] FIG. 10 is a diagram illustrating an example of a progress determination process according to the third embodiment. [Figure 19] 10 is a flowchart showing an example of an operation procedure of an authentication analysis device according to a third embodiment. [Figure 20] A diagram explaining an example of a question example screen [Figure 21] FIG. 10 is a diagram illustrating an example of a user progress notification screen. [Figure 22] FIG. 10 is a diagram illustrating an example of a user progress notification screen. DETAILED DESCRIPTION OF THE INVENTION

[0011] (Background to the first embodiment) Patent Document 1 discloses a communication device that, when it is determined that the total acquisition time length of at least one or more voiceprint data corresponding to a telephone number in a telephone directory and the same telephone number as the acquired telephone number is longer than the time required for voiceprint matching, associates the acquired telephone number with the voiceprint data and stores them. However, since the above-mentioned communication device collectively stores one or more voiceprint data used for voiceprint matching for each speaker, if voiceprint matching is performed using a received voice in which the speaker is emotional, there is a possibility that the speaker's individual characteristics contained in the voiceprint data will vary depending on the speaker's emotions, and the accuracy of the speaker's voiceprint matching will decrease.

[0012] Hereinafter, with reference to the drawings as appropriate, a detailed description of embodiments specifically disclosing a voice registration device and a voice registration method according to the present disclosure will be provided. However, more detailed description than necessary may be omitted. For example, detailed descriptions of already well-known matters and redundant descriptions of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter recited in the claims.

[0013] (Embodiment 1) First, a use case of the voice authentication system 100 according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of a use case of the voice authentication system 100 according to the first embodiment.

[0014] The voice authentication system 100 acquires a voice signal or voice data of a person to be voice authenticated (user US in the example shown in FIG. 1), and registers (stores) the acquired voice signal or voice data for each user US in storage (registered speaker database DB in the example shown in FIG. 1).

[0015] Furthermore, the voice authentication system 100 acquires a voice signal or voice data of a person to be voice authenticated (user US in the example shown in FIG. 1), and compares the acquired voice signal or voice data with a plurality of voice signals or voice data registered (stored) in advance in storage (registered speaker database DB in the example shown in FIG. 1). Based on the comparison result, the voice authentication system 100 evaluates the similarity between the user to be voice authenticated and the voice signal or voice data registered in the storage, and authenticates user US based on the evaluated similarity.

[0016] The voice authentication system 100 according to the first embodiment includes at least an operator-side communication terminal OP1, an authentication analysis device P1, a registered speaker database DB, and an information display unit DP. The authentication analysis device P1 and the registered speaker database DB may be integrally configured. Similarly, the authentication analysis device P1 and the information display unit DP may be integrally configured.

[0017] The voice authentication system 100 shown in Fig. 1 is used, as an example, to authenticate a speaker (user US) in a call center, and authenticates the user US using voice data obtained by collecting the speech of the user US while talking to an operator OP. The voice authentication system 100 shown in Fig. 1 further includes a user-side call terminal UP1 and a network NW. It goes without saying that the overall configuration of the voice authentication system 100 is not limited to the example shown in Fig. 1.

[0018] The user-side call terminal UP1 is connected to the operator-side call terminal OP1 via a network NW so as to be able to communicate wirelessly with the operator-side call terminal OP1. Note that the wireless communication here refers to communication via a wireless LAN (Local Area Network) such as Wi-Fi (registered trademark).

[0019] The user-side communication terminal UP1 is realized by, for example, a notebook PC, a tablet terminal, a smartphone, a telephone, etc. The user-side communication terminal UP1 is a sound collection device equipped with a microphone (not shown), which collects the speech of the user US, converts it into an audio signal, and transmits the converted audio signal to the operator-side communication terminal OP1 via the network NW. The user-side communication terminal UP1 also acquires the audio signal of the speech of the operator OP transmitted from the operator-side communication terminal OP1, and outputs it from a speaker (not shown).

[0020] The network NW is an IP network or a telephone network that connects the user-side call terminal UP1 and the operator-side call terminal OP1 so that voice signals can be transmitted and received. Data is transmitted and received via wired or wireless communication. The wireless communication here refers to communication via a wireless LAN such as Wi-Fi (registered trademark).

[0021] The operator side communication terminal OP1 is connected to the user side communication terminal UP1 and the authentication analysis device P1 via wired or wireless communication so as to be able to transmit and receive data therebetween, and transmits and receives voice signals therebetween.

[0022] The operator-side communication terminal OP1 is realized by, for example, a notebook PC, a tablet terminal, a smartphone, a telephone, etc. The operator-side communication terminal OP1 acquires an audio signal based on the speech of the user US transmitted from the user-side communication terminal UP1 via the network NW, and transmits the audio signal to the authentication analysis device P1. When the operator-side communication terminal OP1 acquires an audio signal including the acquired speech of the user US and the speech of the operator OP, the operator-side communication terminal OP1 may separate the audio signal based on the speech of the user US from the audio signal based on the speech of the operator OP based on audio parameters such as the sound pressure level and frequency band of the audio signal of the operator-side communication terminal OP1. After the separation, the operator-side communication terminal OP1 extracts only the audio signal based on the speech of the user US and transmits it to the authentication analysis device P1.

[0023] The operator terminal OP1 may be communicably connected to each of a plurality of user terminals and simultaneously acquire voice signals from each of the plurality of user terminals. The operator terminal OP1 transmits the acquired voice signals to the authentication analysis device P1. This allows the voice authentication system 100 to simultaneously perform voice authentication processing and voice analysis processing for each of a plurality of users.

[0024] Furthermore, the operator-side communication terminal OP1 may simultaneously acquire a voice signal containing the respective speeches of multiple users. The operator-side communication terminal OP1 extracts a voice signal for each user from the voice signals of the multiple users acquired via the network NW and transmits the voice signal for each user to the authentication analysis device P1. In such a case, the operator-side communication terminal OP1 may analyze the voice signals of the multiple users and separate and extract the voice signals for each user based on voice parameters such as sound pressure level and frequency band. When the voice signals are collected by an array microphone or the like, the operator-side communication terminal OP1 may separate and extract the voice signals for each user based on the direction from which the spoken voices arrive. This allows the voice authentication system 100 to perform voice authentication processing and voice analysis processing for each of the multiple users, even if the voice signals are collected in an environment where multiple users are speaking simultaneously, such as a web conference.

[0025] The authentication analysis device P1, which is an example of a computer, is connected to the operator-side communication terminal OP1, the registered speaker database DB, and the information display unit DP so as to be able to transmit and receive data therebetween. Note that the authentication analysis device P1 may also be connected to the operator-side communication terminal OP1, the registered speaker database DB, and the information display unit DP so as to be able to communicate with each other via a network (not shown) in a wired or wireless manner.

[0026] The authentication analysis device P1 acquires the voice signal of the user US transmitted from the operator-side communication terminal OP1 and detects the speech section in which the user US is speaking from the acquired voice signal. The authentication analysis device P1 identifies the emotion of the user US contained in the voice signal of each detected speech section (hereinafter referred to as "speech voice signal") and extracts the speech voice signal for each identified emotion. The authentication analysis device P1 associates the extracted speech voice signal for each emotion with the telephone number of the user-side communication terminal UP1 and registers (stores) it in the registered speaker database DB.

[0027] If the authentication analysis device P1 determines that the telephone number of the user-side calling terminal UP1 is already registered in the registered speaker database DB, it updates the speech voice signal associated with this telephone number to the acquired new speech voice signal. Also, if the authentication analysis device P1 determines that the telephone number of the user-side calling terminal UP1 is not already registered in the registered speaker database DB, it registers this telephone number in association with the acquired speech voice signal for each emotion.

[0028] In this embodiment 1, an example is shown in which a telephone number is used to identify the user-side calling terminal UP1 (i.e., the user US). However, if the user-side calling terminal UP1 is realized by a PC, a notebook PC, a tablet terminal, etc., or if the user US and the operator OP communicate using a web conferencing system, the IP (Internet Protocol) address of the user-side calling terminal UP1 may be used instead of the telephone number.

[0029] The authentication analysis device P1 also acquires the speech signal of the user US transmitted from the operator-side communication terminal OP1, and performs voice analysis, for example, for each frequency, to extract individual speech features of the user US. The authentication analysis device P1 references the registered speaker database DB and compares the extracted speech features with the speech features of each of multiple users pre-registered in the registered speaker database DB to perform voice authentication of the user US. The authentication analysis device P1 generates an authentication result screen SC including the user authentication result, and transmits it to the information display unit DP for output. It goes without saying that the authentication result screen SC shown in FIG. 1 is merely an example and is not limited to this. The authentication result screen SC shown in FIG. 1 includes the user authentication result message, "The voice matches that of XXXXX."

[0030] The registered speaker database DB, as an example of a database, is a so-called storage, and is configured using a storage medium such as a flash memory, a hard disk drive (HDD), or a solid state drive (SSD). The registered speaker database DB stores (registers) user information of multiple users and speech features in association with each other. The user information here refers to information about the user, such as a user name, a user ID (Identification), or identification information assigned to each user. The registered speaker database DB may be configured integrally with the authentication analysis device P1. The registered speaker database DB associates a speech voice signal for each emotion of the user US with the user US's telephone number, and registers (stores) the associated speech voice signal in a first emotion database DB1, a second emotion database DB2, ..., an Nth emotion database DBN (N: an integer equal to or greater than 1) corresponding to the emotion of the speech voice signal. The number of emotion databases in the registered speaker database DB may be one or more.

[0031] The information display unit DP is configured using, for example, an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence) display, and displays the authentication result screen SC transmitted from the authentication analysis device P1.

[0032] 1, the user-side communication terminal UP1 collects the user US's speech COM12 "This is XXXXX" and the speech COM14 "This is 123245678," converts them into speech signals, and transmits them to the operator-side communication terminal OP1. The operator-side communication terminal OP1 transmits speech signals based on the user US's speech COM12 and COM14 transmitted from the user-side communication terminal UP1 to the authentication analysis device P1.

[0033] When the operator-side communication terminal OP1 acquires audio signals that have been collected from the operator OP's speech COM11 "Please tell me your name," the speech COM13 "Please tell me your membership number," and the user US's speech COM12 and COM14, it separates and removes the audio signals based on the operator OP's speech COM11 and COM13, extracts only the audio signals based on the user US's speech COM12 and COM14, and transmits them to the authentication analysis device P1. This allows the authentication analysis device P1 to improve the accuracy of user authentication by using only the audio signals of the person who is the target of voice authentication.

[0034] An example of the internal configuration of the authentication analysis device P1 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the internal configuration of the authentication analysis device P1 in embodiment 1. The authentication analysis device P1 is configured to include at least a communication unit 20, a processor 210, and a memory 22.

[0035] The communication unit 20, which is an example of an acquisition unit, is connected to the operator-side communication terminal OP1 and the registered speaker database DB so as to be able to communicate data with each other. The communication unit 20 outputs the voice signal transmitted from the operator-side communication terminal OP1 to the processor 210.

[0036] The processor 210 is configured using a semiconductor chip on which at least one of electronic devices such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), a GPU (Graphical Processing Unit), an FPGA (Field Programmable Gate Array), etc. The processor 210 functions as a controller that manages the overall operation of the authentication analysis device P1, and performs control processing for managing the operation of each part of the authentication analysis device P1, data input / output processing between each part of the authentication analysis device P1, data arithmetic processing, and data storage processing.

[0037] Processor 210 realizes the functions of speech section detection unit 21A, speech connection unit 21B, feature extraction unit 21C, similarity calculation unit 21D, emotion identification unit 21E, database registration unit 21F, and progress determination unit 21G by using programs and data stored in ROM (Read Only Memory) 22A of memory 22. Processor 210 uses RAM (Random Access Memory) 22B of memory 22 during operation, and temporarily stores data or information generated or acquired by processor 210 and each unit in RAM 22B of memory 22.

[0038] The processor 210 acquires the telephone number (originator number) of the user-side communication terminal UP1, and collates the acquired telephone number with the telephone numbers of multiple user-side communication terminals registered in the registered speaker database DB to identify the user-side communication terminal UP1 (i.e., user US). If the processor 210 identifies the user-side communication terminal UP1, it executes an update process of the speech voice signal that is registered in the registered speaker database DB and used for user authentication of the user US who is the owner of the user-side communication terminal UP1 (in other words, a registration process of a new speech voice signal), and if the user US cannot be identified, it executes a new registration process of the telephone number of the user US and the speech voice signal that is used for user authentication. The processor 210 generates progress notification screens DP11, DP12A, DP12B (see FIGS. 9 to 11) that notify the operator OP of the progress status of the registration process of the speech voice signal, and outputs them to the information display unit DP.

[0039] The speech section detection unit 21A analyzes the acquired audio signal and detects an utterance section in which the user US is speaking. The speech section detection unit 21A outputs an utterance audio signal corresponding to each utterance section detected from the audio signal to the speech connection unit 21B or the feature extraction unit 21C. The speech section detection unit 21A may also temporarily store the utterance audio signal of each utterance section in the RAM 22B of the memory 22.

[0040] When the speech section detection unit 21A detects two or more speech sections of the same person (user US) from the speech audio signal, the speech connection unit 21B connects the speech audio signals of these speech sections. The speech connection unit 21B outputs the connected speech audio signal to the feature extraction unit 21C. The user authentication method will be described later.

[0041] The feature extraction unit 21C extracts speech features by analyzing the characteristics of an individual's voice, for example, for each frequency, using one or more speech audio signals extracted by the speech segment detection unit 21A. Note that the feature extraction unit 21C may also extract speech features of the speech audio signals output from the speech connection unit 21B. The feature extraction unit 21C associates the extracted speech features with the speech audio signals from which the speech features were extracted, and outputs them to the similarity calculation unit 21D or temporarily stores them in RAM 22B of the memory 22.

[0042] The similarity calculation unit 21D acquires the speech feature of the speech voice signal output from the feature extraction unit 21C. The similarity calculation unit 21D refers to the registered speaker database DB and calculates the similarity between the speech feature of each of the multiple users registered in the registered speaker database DB and the acquired concatenated speech feature. Based on the calculated similarity, the similarity calculation unit 21D identifies the user corresponding to the speech voice signal and performs user authentication.

[0043] If the similarity calculation unit 21D determines that the user has been identified as a result of the user authentication, it generates an authentication result screen SC including information about the identified user (i.e., the authentication result) and outputs it to the information display unit DP via the display I / F (Interface) 23.

[0044] If the similarity calculation unit 21D determines that the calculated similarity is less than a predetermined value, it may determine that user authentication is not possible, and generate and output a control command requesting the speech connection unit 21B to connect speech voice signals. Furthermore, if an upper limit is set for the number of times user authentication is attempted for the same person (user US), and the similarity calculation unit 21D determines that the number of times user authentication is not possible is equal to or exceeds the upper limit, it may generate an authentication result screen (not shown) notifying that user authentication is not possible, and output it to the information display unit DP.

[0045] The emotion identification unit 21E analyzes the speech voice signal detected by the speech segment detection unit 21A using an emotion model stored in the memory 22, and identifies the emotion of the user US corresponding to the speech voice signal (for example, neutral, angry, excited, nervous, depressed, etc.). The emotion identification unit 21E combines the voice utterance signals for each identified emotion and outputs them to the database registration unit 21F. Note that the emotion identification unit 21E may be configured separately from the authentication analysis device P1.

[0046] The database registration unit 21F associates the telephone number of the user-side call terminal UP1 with the voice speech signal for each emotion output from the emotion identification unit 21E, and registers (stores) the correspondence between the telephone number and the voice speech signal for each emotion output from the emotion identification unit 21E in a first emotion database DB1, a second emotion database DB2, ..., an Nth emotion database DBN in the registered speaker database DB.

[0047] The progress determination unit 21G determines the progress of the registration process of the speech voice signal registered in the registered speaker database DB by the database registration unit 21F. The progress of the registration process here refers to the ratio (progress) of the current registration process of the sound types, speech length, etc. of the speech voice signal of the user US registered in the registered speaker database DB to the target registration data amount, such as the number of sound types and speech length (in other words, the length of the speech voice signal), required for user authentication using the speech voice signal.

[0048] The memory 22 includes at least a ROM 22A that stores programs that define the various processes performed by the processor 210 and data used during execution of the programs, and a RAM 22B that serves as a work memory used when the various processes performed by the processor 210 are executed. The ROM 22A stores programs that define the various processes performed by the processor 210 and data used during execution of the programs. The RAM 22B temporarily stores data or information (e.g., speech audio signals before concatenation, speech audio signals after concatenation, speech features corresponding to each speech section before or after concatenation, etc.) generated or acquired by the processor 210.

[0049] The display I / F 23 connects the processor 210 and the information display unit DP so that data communication is possible between them, and outputs the authentication result screen SC generated by the similarity calculation unit 21D of the processor 210 to the information display unit DP.

[0050] Next, the emotion identification process and speech voice signal registration process executed by the authentication analysis device P1 will be described with reference to Fig. 3 and Fig. 4. Fig. 3 is a diagram illustrating an example of emotion identification process in embodiment 1. Fig. 4 is a diagram illustrating an example of registration of speech voice signals. Note that the emotion identification process method described in Fig. 3 and Fig. 4 may also be executed by the emotion identification unit 21E in embodiment 2 in a similar manner.

[0051] 3 and 4, an example of identifying two emotions, "Emotion 1" and "Emotion 2," from a speech audio signal will be described, but the number of identifiable emotions need only be one or more. Furthermore, "Emotion 1" corresponds to the emotion of the speech audio signal registered (stored) in the first emotion database DB1. "Emotion 2" corresponds to the emotion of the speech audio signal registered (stored) in the second emotion database DB2.

[0052] The emotion identification unit 21E extracts sections (time periods) of the speech voice signal that include each of the emotions "Emotion 1" and "Emotion 2" that correspond to each emotion database in the registered speaker database DB.

[0053] 3, the emotion identification unit 21E analyzes the emotion of the user US contained in the speech signal, and classifies the emotion of the speech signal that is equal to or greater than a threshold Th1 as "emotion 2," and the emotion of the speech signal that is less than the threshold Th1 as "emotion 1." The emotion identification unit 21E classifies the speech signal in a time period PR11 from time t11 to time t12 and the speech signal in a time period PR13 from time t13 to time t14 as "emotion 1," and the speech signal in a time period PR12 from time t12 to time t13 as "emotion 2." The emotion identification unit 21E combines the speech signal in the time period PR11 and the speech signal in the time period PR13.

[0054] The database registration unit 21F associates the utterance voice signal for time period PR11 and time period PR13 classified as "emotion 1" with the telephone number of the user US and registers (stores) it in the first emotion database DB1, and associates the utterance voice signal for time period PR12 classified as "emotion 2" with the telephone number of the user US and registers (stores) it in the second emotion database DB2. Here, if the database registration unit 21F determines that there is an utterance voice signal already registered in the first emotion database DB1 or the second emotion database DB2, it overwrites (updates) the registered utterance voice signal with the new utterance voice signal output from the emotion identification unit 21E.

[0055] As a result, the authentication analysis device P1 in the first embodiment can classify the speech voice signals of the user US by emotion and register (store) them in the registered speaker database DB. Therefore, even if the speaker's individual characteristics contained in the speech voice signals vary depending on the emotion of the user US during a call, the authentication analysis device P1 can more effectively prevent a decrease in the accuracy of user authentication by using speech voice signals that are identified as having an emotion similar to or the same as the emotion contained in the acquired speech voice signal in the user authentication process using the registered speech voice signals.

[0056] Furthermore, by overwriting (updating) the speech voice signals registered (stored) in each emotion database with newly acquired speech voice signals, the authentication analysis device P1 can improve the accuracy of user authentication by performing user authentication processing using speech voice signals containing more up-to-date individual characteristics, and more efficiently prevent the occurrence of a shortage of free space in the registered speaker database DB.

[0057] Next, the speech speed conversion process executed by the authentication analysis device P1 will be described with reference to Fig. 5 and Fig. 6. Fig. 5 is a diagram showing an example of a call between an operator OP and a user US. Fig. 6 is a diagram explaining an example of the speech speed conversion process. Note that the example of the speech speed conversion process shown in Fig. 6 is just an example, and the present invention is not limited to this.

[0058] The user-side communication terminal UP1 picks up the user US's speech Us11 "Hello," the speech Us12 "I don't know my PIN," the speech Us13 "My ID is 12345678," and the speech Us14 "My name is XXXXX," converts them into audio signals, and transmits them to the operator-side communication terminal OP1.

[0059] The operator-side communication terminal OP1 collects the operator OP's speech Op11 "How may I help you?", speech Op12 "Yes, please tell me your ID", and speech Op13 "Please tell me your name", converts them into voice signals, and transmits them to the user-side communication terminal UP1. The operator-side communication terminal OP1 also acquires the voice signals transmitted from the user-side communication terminal UP1 and transmits them to the authentication analysis device P1.

[0060] The speech section detection unit 21A in the authentication analysis device P1 detects the speech sections of each of the speech voices Us11 to Us14 of the user US from the voice signal transmitted from the operator-side communication terminal OP1. The speech section detection unit 21A extracts the speech voice signal corresponding to each detected speech section. In FIG. 6, the speech voice signal corresponding to the speech voice Us11 shown in FIG. 5 is represented as "Speech 1," the speech voice signal corresponding to the speech voice Us12 is represented as "Speech 2," the speech voice signal corresponding to the speech voice Us13 is represented as "Speech 3," and the speech voice signal corresponding to the speech voice Us14 is represented as "Speech 4."

[0061] The speech section detection unit 21A converts the speech speed of the speech voice signal by compressing or expanding the speech voice signal so that the speech speed of the detected speech section becomes a predetermined speech speed (for example, the speech speed shown in FIG. 6 = 5.0 characters / second). This allows the authentication analysis device P1 to more accurately calculate the similarity between the speech feature of the speech voice signal used for user authentication and the speech feature of each user registered in the registered speaker database DB, thereby further improving the accuracy of user authentication.

[0062] Hereinafter, with reference to FIG. 6, a specific example of the speech speed conversion process for each of the speech signals "utterance 1" to "utterance 4" of the user US will be described.

[0063] For example, the speech section detection unit 21A performs speech analysis on the speech signal of the user US used for registration (storage) in the registered speaker database DB, and calculates the number of characters contained in the speech signal, the signal length of the speech signal (number of seconds of speech), and the speech rate of the speech signal.

[0064] The speech audio signal "Utterance 1" has a number of characters = 5 characters, a speech time (i.e., speech section) = 0.8 seconds, and a speech rate = 6.25 characters / second. Similarly, the speech audio signal "Utterance 2" has a number of characters = 16 characters, a speech time = 2.9 seconds, and a speech rate = 5.51 characters / second. The speech audio signal "Utterance 3" has a number of characters = 16 characters, a speech time = 4.0 seconds, and a speech rate = 4.0 characters / second. Furthermore, the speech audio signal "Utterance 4" has a number of characters = 12 characters, a speech time = 3.5 seconds, and a speech rate = 3.42 characters / second.

[0065] In such a case, the speech section detection unit 21A converts the speech rate of the speech audio signal of "utterance 1" to a speech time of 1.0 second and a speech rate of 5.0 characters / second, converts the speech rate of the speech audio signal of "utterance 2" to a speech time of 3.2 seconds and a speech rate of 5.0 characters / second, converts the speech rate of the speech audio signal of "utterance 3" to a speech time of 3.2 seconds and a speech rate of 5.0 characters / second, and converts the speech rate of the speech audio signal of "utterance 4" to a speech time of 2.4 seconds and a speech rate of 5.0 characters / second.

[0066] The speech rate of the speech signal may be calculated based on the number of characters and the number of seconds of speech acquired by speech recognition of the speech signal, or may be estimated based on the number of moras, the number of syllables, or the number of sound types and the number of seconds of speech. Alternatively, the speech rate of the speech signal may be estimated by direct arithmetic processing from the time components and frequency components of the speech signal.

[0067] As described above, the authentication analysis device P1 in the first embodiment uses the speech signal converted to a predetermined speech rate to perform processing for identifying the emotion of the user US contained in the speech signal, and processing for registering the speech signal in the registered speaker database DB, thereby realizing the registration of a speech signal that sufficiently includes the individuality required for user authentication processing for each emotion. Therefore, the authentication analysis device P1 can improve the user authentication accuracy in the user authentication process.

[0068] Next, an example of an operation procedure of the authentication analyzing device P1 will be described with reference to Fig. 7 and Fig. 8. Fig. 7 is a flowchart showing an example of an operation procedure of the authentication analyzing device P1 in embodiment 1. Fig. 8 is a flowchart showing an example of an operation procedure of the authentication analyzing device P1 in embodiment 1.

[0069] The communication unit 20 in the authentication analysis device P1 acquires a voice signal (or voice data) of a call with the user side call terminal UP1 transmitted from the operator side call terminal OP1 (St11). The communication unit 20 outputs the acquired voice signal to the processor 210. The communication unit 20 also acquires the telephone number of the user side call terminal UP1 transmitted from the operator side call terminal OP1, and outputs it to the processor 210 (St12). Note that the telephone number here may be the IP address of the user side call terminal UP1, or the like.

[0070] The processor 210 compares the telephone number of the user's calling terminal UP1 output from the communication unit 20 with each of the telephone numbers of multiple user's calling terminals registered (stored) in the registered speaker database DB, determines whether the telephone number of the user's calling terminal UP1 has already been registered in the registered speaker database DB, generates a progress notification screen including the determination result, outputs it to the information display unit DP for display (St12).

[0071] Specifically, if the processor 210 determines in the processing of step St12 that the telephone number of the user-side call terminal UP1 has already been registered in the registered speaker database DB (i.e., there is registration in the registered speaker database DB), it generates a progress notification screen DP12A (see FIG. 10) including information about the user US associated with the telephone number (e.g., the name, address, age, etc. of the user US), and outputs it to the information display unit DP (St12). On the other hand, if the processor 210 determines in the processing of step St12 that the telephone number of the user-side call terminal UP1 has not already been registered in the registered speaker database DB (i.e., there is no registration in the registered speaker database DB), it generates a progress notification screen DP11 (see FIG. 9) notifying that the telephone number has not already been registered, and outputs it to the information display unit DP (St12).

[0072] The processor 210 determines whether or not an operation requesting the start of registration of the speech signal of the user US (that is, a registration start operation) has been accepted by the operator OP (St13).

[0073] If the processor 210 determines in the processing of step St13 that it has received an operation from the operator OP requesting the start of registration of the user US's speech voice signal (i.e., a registration start operation) (St13, YES), it starts registering the user US's speech voice signal in the registered speaker database DB (St14).

[0074] On the other hand, if the processor 210 determines in the processing of step St13 that it has not received an operation from the operator OP requesting the start of registration of the user US's speech voice signal (i.e., a registration start operation) (St13, NO), it returns to the processing of step St13 again and determines whether it has received an operation from the operator OP requesting the start of registration of the user US's speech voice signal (i.e., a registration start operation). If the processor 210 receives a control command from the operator-side call terminal OP1 indicating that the call between the user-side call terminal UP1 and the operator-side call terminal OP1 has ended, it ends the operation procedure shown in Fig. 7.

[0075] The speech period detection unit 21A in the processor 210 detects a speech period in which the user US is speaking from the voice signal of the user US transmitted from the operator-side communication terminal OP1 (St15).

[0076] The processor 210 performs voice recognition on the number of characters included in the speech voice signal corresponding to the speech section. The voice section detection unit 21A calculates the speech rate of the speech voice signal based on the number of voice-recognized characters and the signal length of the speech voice signal (speech length, number of seconds of speech, etc.). The processor 210 performs a speech rate conversion process on the speech voice signal to convert the speech rate of the speech voice signal to a predetermined speech rate (St16). Note that the process of step St16 is not essential and may be omitted.

[0077] The processor 210 records information about the detected speech section (e.g., the start and end times of the speech section, the number of characters, the signal length (speech audio length, number of seconds of speech, etc.), the speech speed before or after speech speed conversion, etc.) in the memory 22 (St17).

[0078] The processor 210 analyzes and identifies the emotion of the user US contained in the speech voice signal (St18). The processor 210 connects the identified speech voice signals by emotion (St19).

[0079] The processor 210 determines (calculates) the progress of the current registration process of the speech voice signal with respect to the target amount of registration data to be registered in the registration speaker database DB (St20).

[0080] The processor 210 generates registration progress information IN13 indicating the progress of the current registration process of the speech voice signal. The processor 210 generates progress notification screens DP11 and DP12A (see FIGS. 9 and 10) including the registration progress information IN13, outputs them to the information display unit DP for display, and notifies the operator OP of the progress determination result in real time (St21).

[0081] The processor 210 determines whether the current progress is equal to or greater than a threshold value (St22).

[0082] When the processor 210 determines in the processing of step St22 that the current progress is equal to or greater than the threshold (St22, YES), it determines whether the user side call terminal UP1 has a telephone number registered in the registered speaker database DB (St23).

[0083] On the other hand, when it is determined in the process of step St22 that the current progress degree is not equal to or greater than the threshold (St22, NO), the processor 210 determines whether or not a control command requesting continuation of the registration process has been received (St24).

[0084] If the processor 210 determines in the processing of step St24 that it has received a control command requesting that the registration process be continued based on a control command based on an operator OP operation sent from the operator side call terminal OP1 (St24, YES), it proceeds to the processing of step St15.

[0085] On the other hand, if the processor 210 determines in the processing of step St24 that it has not received a control command requesting the continuation of the registration process based on the control command based on the operator OP operation sent from the operator-side call terminal OP1 (i.e., it has received a control command requesting the cancellation or termination of the registration process) (St24, NO), it proceeds to the processing of step St30 and cancels the registration process of the acquired speech voice signal to the registered speaker database DB (i.e., registration failed) (St30).

[0086] If the progress determination unit 21G determines in the processing of step St23 that the user side call terminal UP1 has a telephone number registered in the registered speaker database DB (St23, YES), the processor 210 calculates the similarity between the speech voice signal registered in the registered speaker database DB and associated with the matched telephone number and the speech voice signal acquired by the call (St25). Here, the processor 210 calculates the similarity between the speech voice signal linked by emotion by the emotion identification unit 21E and the speech voice signal registered in the registered speaker database DB and containing an emotion similar to or the same as the emotion contained in the speech voice signal.

[0087] On the other hand, if the progress determination unit 21G determines in the processing of step St23 that the telephone number is not that of the user's calling terminal UP1 whose telephone number is registered in the registered speaker database DB (St23, NO), the processor 210 associates the speech voice signal linked by emotion with the acquired telephone number of the user's calling terminal UP1, and registers (stores) it in each emotion database of the registered speaker database DB (St26), and completes the registration process of the speech voice signal (St27).

[0088] The processor 210 determines whether the calculated similarity is equal to or greater than a threshold value (St28).

[0089] If the processor 210 determines in the processing of step St28 that the calculated similarity is equal to or greater than the threshold value (St28, YES), it determines whether to update (overwrite) the speech voice signal associated with the telephone number of the user side call terminal UP1 currently registered (stored) in the registered speaker database DB with the newly acquired speech voice signal by accepting an operator OP operation (St29).

[0090] On the other hand, if the processor 210 determines in the processing of step St28 that the calculated similarity is not greater than the threshold value (St28, NO), it determines that the speech voice signal registered in the registered speaker database DB and the acquired speech voice signal are the voices of different users, and cancels the process of registering the acquired speech voice signal in the registered speaker database DB (i.e., registration failed) (St30).

[0091] After the process of step St27 or step St30, the processor 210 generates a registration result screen (not shown) including the result of the registration process of the speech sound signal to the registered speaker database DB, and outputs it to the information display unit DP for display (St31).

[0092] As described above, the authentication analysis device P1 in the first embodiment can classify the speech voice signals of the user US by emotion and register (store) them in the registered speaker database DB. Therefore, even if the speaker's individual characteristics contained in the speech voice signals vary depending on the emotion of the user US during a call, the authentication analysis device P1 can further improve the accuracy of user authentication by using speech voice signals that are identified as having an emotion similar to or the same as the emotion contained in the acquired speech voice signal in the user authentication process using the registered speech voice signals.

[0093] Next, a progress notification screen for the registration process of an utterance voice signal acquired from a user-side call terminal UP1 (i.e., user US) that is not registered in the registered speaker database DB will be described with reference to Fig. 9. Fig. 9 is a diagram illustrating an example of a progress notification screen DP11 for a new user. It goes without saying that the progress notification screen DP11 shown in Fig. 9 is an example and is not limited to this.

[0094] The progress notification screen DP11 is a screen for notifying the operator OP of the progress of the registration process of the speech voice signal of the unregistered user-side call terminal UP1 (i.e., the user US), and is generated by the processor 210 and displayed by the information display unit DP. The progress notification screen DP11 includes a user information field IN11, a registration name input field IF11, a voice signal graph IN12, registration progress information IN13, and a speaker registration button BT11.

[0095] The user information field IN11 includes information about the user US that is registered (stored) in association with the telephone number, IP address, etc. of the user-side call terminal UP1. The user information field IN11 shown in Fig. 9 is generated to include information such as, for example, the caller number "xx-xxx-xxx", such as the telephone number, IP address, etc. of the user-side call terminal UP1, registration presence / absence information "not registered (new)" indicating whether the telephone number of the user-side call terminal UP1 has been registered, the name of the user US "none", the address of the user US "none", the age of the user US "none", and speaker registration presence / absence information "none".

[0096] The registered name input field IF11 is an input field that can accept an operator's operation for a user name to be registered in the registered speaker database DB. Note that the registered name input field IF11 may be capable of inputting information about the user US included in the user information field IN11.

[0097] The voice signal graph IN12 displays the result of identifying the emotion of the user US contained in the voice signal SG12 superimposed on the voice signal SG12 of the detected speech section.

[0098] The registration progress information IN13 includes the progress IN131 of the current registration process of the speech voice signal relative to the target registration data amount to be registered in the registration speaker database DB, and the number of sound types "X" contained in the speech voice signal and the speech length (signal length) "X".

[0099] The speaker registration button BT11 accepts an operation to start or stop the registration process of an utterance voice signal in the registered speaker database DB by the operator OP. When the speaker registration button BT11 accepts an operation to start the registration process of an utterance voice signal in the registered speaker database DB by the operator OP operation, it executes the registration process of an utterance voice signal acquired from the user-side call terminal UP1 (i.e., the user US) that is not registered in the registered speaker database DB. When the speaker registration button BT11 accepts an operation to stop the registration process of an utterance voice signal in the registered speaker database DB by the operator OP operation, it stops the registration process of an utterance voice signal acquired from the user-side call terminal UP1 that is not registered in the registered speaker database DB.

[0100] Next, a progress notification screen for the registration process of the speech voice signal of the user-side call terminal UP1 (i.e., the user US) registered in the registered speaker database DB will be described with reference to Fig. 10 and Fig. 11. Fig. 10 is a diagram illustrating an example of a progress notification screen DP12A for a registered user. Fig. 11 is a diagram illustrating an example of a progress notification screen DP12B for a registered user. It goes without saying that the progress notification screen DP12A shown in Fig. 10 and the progress notification screen DP12B shown in Fig. 11 are merely examples, and the present invention is not limited to these.

[0101] The progress notification screen DP12A is a screen for notifying the operator OP of the progress of the registration process of the speech voice signal acquired from the user-side call terminal UP1 (i.e., the user US) registered in the registered speaker database DB, and is generated by the processor 210 and displayed by the information display unit DP. The progress notification screen DP12A includes a user information field IN14, a registration name input field IF11, a voice signal graph IN15, registration progress information IN13, and a speaker registration button BT12.

[0102] The user information field IN14 includes information about the user US that is registered (stored) in association with the telephone number, IP address, etc. of the user-side call terminal UP1. The user information field IN14 shown in Fig. 10 is generated by including information such as a caller number "xxx-xxx-xxx", such as the telephone number, IP address, etc. of the user-side call terminal UP1, registration presence / absence information "Registered" indicating whether the telephone number of the user-side call terminal UP1 has been registered, the name of the user US "xxx", the address of the user US "xxx", the age of the user US "xxx", and speaker registration presence / absence information "Yes".

[0103] The voice signal graph IN15 displays the result of identifying the emotion of the user US contained in the voice signal SG13 by superimposing it on the voice signal SG13 of the detected speech section.

[0104] The speaker registration button BT12 accepts an operation by the operator OP to start or stop the registration process of the speech voice signal in the registered speaker database DB. When the speaker registration button BT12 accepts an operation by the operator OP to start the registration process of the speech voice signal in the registered speaker database DB, it updates (overwrites) the speech voice signal currently registered in the registered speaker database DB with the acquired speech voice signal. When the speaker registration button BT12 accepts an operation by the operator OP to stop the registration process of the speech voice signal in the registered speaker database DB, it stops the registration process of the speech voice signal in the registered speaker database DB.

[0105] The progress notification screen DP12B is a screen that is generated by the processor 210 and displayed by the information display unit DP when it is determined that the speech sound signal acquired by the progress degree determination unit 21G has reached the registered target amount (the progress degree is equal to or greater than a threshold value). The progress notification screen DP12B is generated to include new update notification information IN16 in addition to the various information included in the progress notification screen DP12A.

[0106] Here, the update notification information IN16 may be generated when it is determined that the speech voice signal acquired by the progress determination unit 21G has reached the registration target amount (the progress is above a threshold), and the speaker registration button BT11 may be generated when an operation to start the registration process of the speech voice signal to the registered speaker database DB is accepted by an operator OP operation.

[0107] The update notification information IN16 includes a message MSG11 "Emotion 1 for XX has already been registered and it has been confirmed that the speaker is the same. Do you want to update the database?", an update button BT13, and a registration cancellation button BT14.

[0108] When the update button BT13 is pressed (selected) by the operator OP operation, the processor 210 updates the speech voice signal already registered in the registered speaker database DB with the acquired speech voice signal.

[0109] As described above, the authentication analysis device P1 in embodiment 1 includes a communication unit 20 (an example of an acquisition unit) that acquires an audio signal of a speaker's (i.e., a user's) spoken voice, an emotion identification unit 21E that identifies at least one emotion of the speaker contained in the voice signal, and a database registration unit 21F (an example of a registration unit) that registers the audio signal for each emotion (i.e., the spoken voice signal) in a registered speaker database DB (an example of a database) based on the identification result by the emotion identification unit 21E.

[0110] As a result, the authentication analysis device P1 in the first embodiment can classify the speech voice signals of the user US by emotion and register (store) them in the registered speaker database DB. Therefore, even if the speaker's individual characteristics contained in the speech voice signals vary depending on the emotion of the user US during a call, the authentication analysis device P1 can more effectively prevent a decrease in the accuracy of user authentication by using speech voice signals that are identified as having an emotion similar to or the same as the emotion contained in the acquired speech voice signal in the user authentication process using the registered speech voice signals.

[0111] Furthermore, as described above, the registered speaker database DB in the authentication analysis device P1 in embodiment 1 includes a plurality of emotion databases corresponding to each emotion (for example, the first emotion database DB1, the second emotion database DB2, ..., the Nth emotion database DBN shown in FIG. 2). The database registration unit 21F registers the voice signal for each emotion in the emotion database corresponding to the emotion. This allows the authentication analysis device P1 in embodiment 1 to manage voice signals for each emotion, and therefore can use the registered spoken voice signal in the user authentication process by referring to the emotion database corresponding to the emotion of the user US during a call. Therefore, the authentication analysis device P1 can improve the accuracy of user authentication.

[0112] Furthermore, as described above, the communication unit 20 in the authentication analysis device P1 in embodiment 1 acquires an uttered voice and speaker information that can identify the speaker of the uttered voice. The database registration unit 21F registers the voice signal for each emotion and the speaker information in association with each other in an emotion database corresponding to the emotion. This allows the authentication analysis device P1 in embodiment 1 to manage voice signals of the same speaker even when voice signals of each emotion are registered in each emotion database.

[0113] Furthermore, as described above, when the database registration unit 21F in the authentication analysis device P1 in embodiment 1 determines that the same speaker information as the acquired speaker information exists among the speaker information registered in the emotion database, it updates the voice signal of the same speaker information that has already been registered in the emotion database to a voice signal identified for each emotion. In this way, the authentication analysis device P1 in embodiment 1 can more efficiently prevent the occurrence of a shortage of free space in the registered speaker database DB by overwriting (updating) the speech voice signal registered (stored) in each emotion database with the newly acquired speech voice signal.

[0114] As described above, the authentication analysis device P1 in embodiment 1 further includes a similarity calculation unit 21D that, when the database registration unit 21F determines that the same speaker information as the acquired speaker information is found among the speaker information registered in the emotion database, calculates the similarity between the voice signal of the same speaker information and the acquired voice signal. When the database registration unit 21F determines that the determined similarity is equal to or greater than a threshold, it updates the voice signal already registered in the emotion database with the identified voice signal. As a result, the authentication analysis device P1 in embodiment 1 can associate speaker information determined to be the same speaker based on the similarity, and overwrite (update) the uttered voice signal registered (stored) in each emotion database with the newly acquired uttered voice signal.

[0115] Furthermore, as described above, when the database registration unit 21F in the authentication analysis device P1 in embodiment 1 determines that the calculated similarity is not equal to or greater than the threshold, it omits updating of the voice signals already registered in the emotion database. As a result, the authentication analysis device P1 in embodiment 1 can omit overwriting (updating) the speech voice signals registered (stored) in each emotion database when it is determined that the persons are not the same based on the similarity.

[0116] Furthermore, as described above, when the database registration unit 21F in the authentication analysis device P1 in embodiment 1 determines that there is no speaker information identical to the acquired speaker information among the speaker information registered in the emotion database, it registers the voice signal identified for each emotion in the corresponding emotion database. As a result, when the authentication analysis device P1 in embodiment 1 determines based on the similarity that no speech signal of the same speaker is registered, it can register (store) the acquired speech signal for each emotion in each emotion database as a new speech signal of this speaker.

[0117] As described above, the authentication analysis device P1 in the first embodiment further includes a speech interval detection unit 21A (an example of a conversion unit) that converts the voice signal acquired by the communication unit 20 into a predetermined speech speed. The speech interval detection unit 21A identifies emotions using the voice signal converted into the predetermined speech speed. As a result, the authentication analysis device P1 in the first embodiment can improve the accuracy of user authentication by converting the speech speed into the predetermined speech speed even when there is variation in the speech speed of speakers.

[0118] As described above, in the authentication analysis device P1 according to the first embodiment, the speaker information is the telephone number of the user-side communication terminal UP1 (an example of a sound collection device) that collects the uttered voice. This allows the authentication analysis device P1 according to the first embodiment to more effectively prevent impersonation by other speakers.

[0119] (Background to the second embodiment) Patent Document 1 discloses a communication device that stores an acquired telephone number and voiceprint data in association with each other when it determines that the total acquisition time length of at least one or more voiceprint data corresponding to the same telephone number as a telephone number in a telephone directory and the acquired telephone number is longer than the time required for voiceprint matching. However, even with the same telephone number, for example, there are cases where the speaker corresponding to the telephone number and voiceprint data other than that speaker are spoken interchangeably. In such cases, the communication device stores voiceprint data of multiple different speakers in association with the acquired telephone number, which could result in a decrease in voiceprint matching accuracy during voiceprint matching.

[0120] Therefore, in the following embodiment 2, an example of a speech voice registration device and a speech voice registration method that detects a change in speaker during a call and collects speech voices to be used for voice authentication will be described. Note that in the following description, the same components as those in embodiment 1 will be assigned the same reference numerals, and their description will be omitted.

[0121] (Embodiment 2) A block diagram showing an example of the internal configuration of the authentication analysis device P1A in the second embodiment will be described with reference to Fig. 12. Fig. 12 is a block diagram showing an example of the internal configuration of the authentication analysis device P1A in the second embodiment.

[0122] The authentication analysis device P1A in the second embodiment is an example of a computer that detects a change of speaker during a call and extracts, from the speech signal, the speech signal of the user US to be registered in the registered speaker database DB. The authentication analysis device P1A analyzes and identifies emotions contained in the extracted speech signal of the user US, and registers (stores) the speech signal for each identified emotion in the corresponding emotion database.

[0123] The processor 211 of the authentication analysis device P1A in embodiment 2 realizes the functions of each of the speech section detection unit 21A, the speech connection unit 21B, the feature extraction unit 21C, the similarity calculation unit 21D, the emotion identification unit 21E, the database registration unit 21F, the progress determination unit 21G, and the speaker switching identification unit 21H.

[0124] The speaker switching identification unit 21H generates a most recently connected signal by connecting each of a plurality of most recently detected speech voice signals transmitted from the operator-side call terminal OP1 and detected by the speech period detection unit 21A. The speaker switching identification unit 21H determines the similarity between the generated most recently connected signal and the speech voice signal of the user US registered in the registered speaker database DB. The speaker switching identification unit 21H repeatedly executes the process of generating the most recently connected signal and the process of determining the similarity, and determines a speaker switch based on the determined similarity or a change in the similarity.

[0125] The speaker change identification unit 21H extracts the speech voice signal of the user US whose speech voice signal is to be registered in the registered speaker database DB based on the timing at which the speaker change is determined, generates a long-term concatenated signal by concatenating all of the extracted speech voice signals, and outputs the signal to the database registration unit 21F.

[0126] The database registration unit 21F registers (stores) the speech voice signal in each emotion database of the registered speaker database DB based on the long-time connected signal output from the speaker switching identification unit 21H and the emotion identification results of the speech voice signal corresponding to each speech section output from the emotion identification unit 21E.

[0127] Next, the speaker switching identification process executed by the speaker switching identification unit 21H will be described with reference to Fig. 13. Fig. 13 is a diagram illustrating an example of the speaker switching identification process. Note that the speaker switching identification process described in Fig. 13 is just an example and is not limited to this. For example, the number of speech voice signals included in the most recently connected signal is not limited to three. Note that in the description of Fig. 13, the owner of the user-side call terminal UP1 is user A.

[0128] 13, user A, who is the owner of user-side communication terminal UP1, hands over user-side communication terminal UP1 to user B while talking to operator OP. As a result, the voice signal transmitted from operator-side communication terminal OP1 switches from the voice of user A to the voice of user B.

[0129] Each of the speech audio signals Us21, Us22, Us23, Us24, and Us25 is a speech audio signal of user A. "Utterance A1" to "Utterance A5" corresponding to each of the speech audio signals Us21 to Us25 indicate the speech audio signal of each speech section detected by the speech section detection unit 21A.

[0130] Each of the speech audio signals Us31, Us32, Us33, and Us34 is a speech audio signal of user B. "Utterance B1" to "Utterance B4" corresponding to each of the speech audio signals Us31 to Us34 indicate a speech audio signal detected by the speech period detection unit 21A.

[0131] The speech section detection unit 21A in the authentication analysis device P1 detects the speech sections "utterance A1" to "utterance A5" and "utterance B1" to "utterance B4" of users A and B, respectively, from the voice signal transmitted from the operator-side communication terminal OP1.

[0132] The processor 211 starts registering the user A's speech signal in the registered speaker database DB when it determines that it has received an operation from the operator OP requesting the start of registration of the user A's speech signal (i.e., a registration start operation).

[0133] The processor 211 compares the telephone number of the user-side telephone terminal UP1 sent from the operator-side telephone terminal OP1 with the telephone numbers of each of the multiple user-side telephone terminals registered in the registered speaker database DB, and identifies user A, the owner of the user-side telephone terminal UP1.

[0134] The speaker switching identification unit 21H determines the similarity between a most recently connected signal ML11, which connects the three speech sections "Utterance A2" to "Utterance A4" most recently detected by the speech section detection unit 21A, and the speech voice signal of user A registered in the registered speaker database DB. At this time, since the most recently connected signal ML11 is entirely composed of the speech voice of user A and the similarity is equal to or greater than a predetermined threshold, the speaker switching identification unit 21H determines the similarity of the most recently connected signal ML11 to be "high." The speaker switching identification unit 21H records the most recently connected signal ML11 used in the first similarity determination and the similarity determined the first time in memory 22 in association with each other.

[0135] The speaker switching identification unit 21H determines the similarity between a most recently connected signal ML12, which connects the three speech sections "Utterance A3" to "Utterance A5" most recently detected by the speech section detection unit 21A, and the speech voice signal of user A registered in the registered speaker database DB. At this time, since the most recently connected signal ML12 is composed entirely of the speech voice of user A and the similarity is equal to or greater than a predetermined threshold, the speaker switching identification unit 21H determines the similarity of the most recently connected signal ML12 to be "high." The speaker switching identification unit 21H records the most recently connected signal ML12 used in the second similarity determination and the similarity determined the second time in memory 22 in association with each other.

[0136] The speaker switching identification unit 21H determines whether the speaker of the user-side call terminal UP1 has switched based on the change in the similarity determined by the first similarity determination process and the similarity determined by the second similarity determination process. Because the similarity of the most recent linked signal ML11 is "high" and the similarity of the most recent linked signal ML12 is also "high," the speaker switching identification unit 21H in Fig. 13 determines that the speaker of the most recent linked signal ML11 and the speaker of the most recent linked signal ML12 have not switched.

[0137] The speaker switching identification unit 21H determines the similarity between a most recently connected signal ML13, which is a combination of the three most recently detected speech periods, "Utterance A4" to "Utterance B1," detected by the speech period detection unit 21A, and the speech signal of user A registered in the registered speaker database DB. At this time, since the most recently connected signal ML13 is composed of speech periods of user A and user B and the similarity is less than the predetermined threshold, the speaker switching identification unit 21H determines the similarity of the most recently connected signal ML13 to be "low." The speaker switching identification unit 21H records the most recently connected signal ML13 used in the third similarity determination and the similarity determined the third time in memory 22 in association with each other.

[0138] The speaker switching identification unit 21H determines whether the speaker of the user-side call terminal UP1 has switched based on the change in the similarity determined by the second similarity determination process and the similarity determined by the third similarity determination process. Because the similarity of the most recent linked signal ML12 is "high" and the reliability of the most recent linked signal ML13 is "low," the speaker switching identification unit 21H in Fig. 13 determines that the speaker of the most recent linked signal ML12 has switched from the speaker of the most recent linked signal ML13 to the speaker of the most recent linked signal ML13.

[0139] The speaker switching identification unit 21H determines the similarity between a most recently concatenated signal ML14, which concatenates the three speech sections, "Utterance A5" to "Utterance B2," most recently detected by the speech section detection unit 21A, and the speech voice signal of user A registered in the registered speaker database DB. At this time, since the most recently concatenated signal ML14 is composed of speech voices of user A and user B and the similarity is less than the predetermined threshold, the speaker switching identification unit 21H determines the reliability of the most recently concatenated signal ML14 to be "low." The speaker switching identification unit 21H records the most recently concatenated signal ML14 used in the fourth similarity determination and the similarity determined the fourth time in memory 22 in association with each other.

[0140] The speaker switching identification unit 21H determines whether the speaker of the user-side call terminal UP1 has switched based on the change in the similarity determined by the third similarity determination process and the similarity determined by the fourth similarity determination process. Because the similarity of the most recent linked signal ML13 is "low" and the reliability of the most recent linked signal ML14 is also "low," the speaker switching identification unit 21H in Fig. 13 determines that the speaker of the most recent linked signal ML13 and the speaker of the most recent linked signal ML14 have not switched.

[0141] The speaker switching identification unit 21H determines the similarity between a most recently concatenated signal ML15, which concatenates the three speech sections "Utterance B1" to "Utterance B3" most recently detected by the speech section detection unit 21A, and the speech signal of user A registered in the registered speaker database DB. At this time, since the most recently concatenated signal ML15 is entirely composed of the speech of user B and the similarity is less than the predetermined threshold, the speaker switching identification unit 21H determines the similarity of the most recently concatenated signal ML15 to be "low." The speaker switching identification unit 21H records the most recently concatenated signal ML15 used in the fifth similarity determination and the similarity determined the fifth time in memory 22 in association with each other.

[0142] The speaker switching identification unit 21H determines whether the speaker of the user-side call terminal UP1 has switched based on the change in the similarity determined by the fourth similarity determination process and the similarity determined by the fifth similarity determination process. Because the similarity of the most recent linked signal ML14 is "low" and the similarity of the most recent linked signal ML15 is "low," the speaker switching identification unit 21H in Fig. 13 determines that the speaker of the most recent linked signal ML14 and the speaker of the most recent linked signal ML15 have not switched.

[0143] The speaker switching identification unit 21H determines the similarity between a most recently concatenated signal ML16, which concatenates the three speech periods "Utterance B2" to "Utterance B4" most recently detected by the speech period detection unit 21A, and the speech signal of user A registered in the registered speaker database DB. At this time, since the most recently concatenated signal ML16 is entirely composed of the speech of user B and the similarity is less than the predetermined threshold, the speaker switching identification unit 21H determines the similarity of the most recently concatenated signal ML16 to be "low." The speaker switching identification unit 21H records the most recently concatenated signal ML16 used in the sixth similarity determination and the similarity determined the sixth time in association with each other in the memory 22.

[0144] The speaker switching identification unit 21H determines whether the speaker of the user-side call terminal UP1 has switched based on the change in the similarity determined by the fifth similarity determination process and the similarity determined by the sixth similarity determination process. Because the similarity of the most recent linked signal ML15 is "low" and the similarity of the most recent linked signal ML16 is also "low," the speaker switching identification unit 21H in Fig. 13 determines that the speaker of the most recent linked signal ML15 and the speaker of the most recent linked signal ML16 have not switched.

[0145] Based on the determined similarity, the speaker change identification unit 21H generates a long-term concatenated signal LT12 by concatenating each of the speech voice signals Us21 to Us25 up to the most recent concatenated signal ML12 determined to be user A, and outputs the signal to the database registration unit 21F.

[0146] The database registration unit 21F associates the long-term connection signal LT12 output from the speaker switching identification unit 21H with the telephone number of the user side call terminal UP1 based on the emotion identification result for each speech voice signal output from the emotion identification unit 21E, and registers (stores) the association in each emotion database of the registered speaker database DB.

[0147] In addition, the speaker change identification unit 21H in embodiment 2 may generate long-term concatenated signals LT11, LT12, LT13, LT14, LT15, ​​and LT16 each time it executes the similarity determination process, by concatenating all speech audio signals from the speech section when the similarity determination process is started to the most recent speech section used in the similarity determination process.

[0148] In addition, if the speaker change identification unit 21H determines that the similarity is greater than or equal to a predetermined value and that the speaker has not changed, it may generate a long-term concatenated signal by concatenating all speech audio signals up to this speech section and output it to the database registration unit 21F.

[0149] Next, the emotion identification processing in the second embodiment will be described with reference to Fig. 14. Fig. 14 is a diagram illustrating an example of the emotion identification processing in the second embodiment. Note that the emotion identification processing method described in Fig. 14 may also be executed by the emotion identification unit 21E in the first embodiment in a similar manner.

[0150] 14 illustrates, as an example, an example of distinguishing between a speech signal containing an emotion (i.e., abnormal) and a speech signal containing no emotion (i.e., normal), but the present invention is not limited to this. For example, the emotion identification unit 21E may distinguish between a speech signal containing a predetermined emotion (e.g., anger) and a speech signal containing no predetermined emotion, or may distinguish between a speech signal containing two or more predetermined emotions (e.g., anger, excitement) and a speech signal containing no such two or more predetermined emotions. The emotion identification unit 21E may also perform further emotion identification processing on the speech signal containing no such two or more predetermined emotions to identify the emotion corresponding to each emotion database.

[0151] The emotion identification unit 21E identifies the emotion of the user US from the acquired speech voice signal SG21, and extracts a time period PR22 of the speech voice signal containing emotion and time periods PR21 and PR23 of the speech voice signal not containing emotion.

[0152] 14, the emotion identification unit 21E analyzes the emotion of the user US contained in the speech voice signal SG21, and determines that a speech voice signal equal to or greater than a threshold Th2 contains an emotion, and determines that a speech voice signal less than a threshold Th1 does not contain an emotion. The emotion identification unit 21E classifies the speech voice signal in time period PR21 from time t41 to time t42 and the speech voice signal in time period PR23 from time t43 to time t44 as "no emotion," and the speech voice signal in time period PR22 from time t42 to time t43 as "emotion present." The emotion identification unit 21E combines the speech voice signal in time period PR21 and the speech voice signal in time period PR23, and outputs the combined results to the database registration unit 21F.

[0153] The database registration unit 21F associates the speech voice signal output from the emotion identification unit 21E with the telephone number of the user-side call terminal UP1, and registers (stores) the result in the registered speaker database DB.

[0154] As a result, the authentication analysis device P1 in the second embodiment can extract only the speech voice signals (speech voice signals not including emotions in FIG. 14) that are more suitable for user authentication processing based on the emotions included in the speech voice signals of the user US, and register (store) them in the registered speaker database DB. Therefore, even if the speaker's individual characteristics included in the speech voice signals vary depending on the emotions of the user US during a call, the authentication analysis device P1 can more effectively suppress a decrease in user authentication accuracy by using the speech voice signals not including emotions in the user authentication processing using the registered speech voice signals.

[0155] Furthermore, by overwriting (updating) the speech voice signal registered (stored) in the registered speaker database DB with the newly acquired speech voice signal, the authentication analysis device P1 can improve the accuracy of user authentication by performing user authentication processing using a speech voice signal containing more up-to-date individual characteristics, and more efficiently prevent the occurrence of a shortage of free space in the registered speaker database DB.

[0156] Next, the operation procedure of the authentication analysis device P1A in the second embodiment will be described with reference to Fig. 15 and Fig. 16, respectively. Fig. 15 is a flowchart explaining an example of a voice registration process in the second embodiment. Fig. 16 is a flowchart explaining an example of a voice registration process in the second embodiment. In the explanation of Fig. 15 and Fig. 16, explanation of the same processes as those in the operation procedure of the authentication analysis device P1 in the first embodiment shown in Fig. 7 and Fig. 8 will be omitted.

[0157] The processor 211 of the authentication analysis device P1A extracts speech features contained in the speech audio signal of each utterance section (St32).

[0158] The processor 211 generates a most recently concatenated signal by concatenating the respective speech voice signals of the most recent several speech periods detected by the speech period detection unit 21A. The processor 211 compares the telephone number of the user-side call terminal UP1 with the telephone numbers of the multiple user-side call terminals registered in the registered speaker database DB. The processor 211 determines the similarity between the generated most recently concatenated signal and the speech voice signals of each emotion associated with the same telephone number as the telephone number of the user-side call terminal UP1 in the registered speaker database DB as a result of the comparison. The processor 211 determines whether the speaker has switched (speaker switching identification) based on a change in the determined similarity (St33). Note that here, the processor 211 may omit the speaker switching identification process if this is the first time the similarity determination process is being performed.

[0159] The processor 211 associates the speech voice signal whose similarity is equal to or greater than the threshold with the telephone number of the user-side call terminal UP1, and registers (stores) the associated speech voice signal in each emotion database of the registered speaker database DB (St34). Note that, if the processor 211 determines in the processing of step St34 that the determined similarity is not equal to or greater than the threshold, it omits the processing of registering (storing) the speech voice signal in each emotion database of the registered speaker database DB.

[0160] The processor 211 determines whether or not the speech voice signal of the target data amount for registration has been registered in the registration speaker database DB (St35).

[0161] In the process of step St35, when it is determined that the registration of the speech voice signal of the registration target data amount in the registration speaker database DB is completed (St35, YES), the processor 211 completes the registration process of the speech voice signal (St36).

[0162] If the processor 211 determines in the processing of step St35 that the speech voice signal with the target data amount for registration has not been completely registered in the registration speaker database DB (St35, NO), it determines whether or not a control command requesting that the registration process be continued has been received (St37).

[0163] If the processor 211 determines in the processing of step St37 that it has received a control command requesting that the registration process be continued based on a control command based on an operator OP operation sent from the operator side call terminal OP1 (St37, YES), it proceeds to the processing of step St15.

[0164] On the other hand, if the processor 211 determines in the processing of step St37 that it has not received a control command requesting the continuation of the registration process based on the control command based on the operator OP operation transmitted from the operator-side call terminal OP1 (i.e., a control command requesting the cancellation or termination of the registration process has been received) (St37, NO), it cancels the registration process of the acquired speech voice signal to the registered speaker database DB (i.e., registration failed) (St39).

[0165] After processing step St36 or step St39, the processor 210 generates a registration result screen (not shown) including the registration process result of the speech voice signal in the registered speaker database DB, and outputs it to the information display unit DP for display (St38).

[0166] As described above, the authentication analysis device P1 in the second embodiment can register only speech signals that are more suitable for user authentication processing in the registered speaker database DB based on the presence or absence of emotions contained in the speech signal of the user US. Therefore, even if the speaker's individual characteristics contained in the speech signal vary depending on the emotion of the user US during the call, the authentication analysis device P1 can further improve the accuracy of user authentication in user authentication processing using the registered speech signals.

[0167] As described above, the authentication analysis device P1A in embodiment 2 includes a communication unit 20 (an example of an acquisition unit) that acquires an audio signal of a speaker's speech, a speech section detection unit 21A (an example of a detection unit) that detects a first speech section of the speaker and a second speech section different from the first speech section from the audio signal, a speaker switching identification unit 21H (an example of a detection unit) that compares the audio signal of the first speech section with the audio signal of the second speech section to detect a switch from one speaker to another speaker different from the speaker (for example, a switch from user A to user B shown in FIG. 13), and a database registration unit 21F (an example of a registration unit) that registers the speaker's audio signal in a registered speaker database DB (an example of a database) based on the detection of the switch by the speaker switching identification unit 21H.

[0168] As a result, the authentication analysis device P1A in embodiment 2 can detect speaker switching based on the speech voice signal of each speech section, even when speech signals of different speakers are acquired during a call using the same user-side call terminal UP1. Therefore, the authentication analysis device P1A can more effectively prevent erroneous registration, such as when a speech voice signal registered in the registered speaker database DB is mixed with or overwritten by a speech voice signal of another speaker.

[0169] As described above, the authentication analysis device P1A in the second embodiment further includes a similarity calculation unit 21D that calculates the similarity between two different voice signals. The communication unit 20 further acquires speaker information that can identify the speaker. The similarity calculation unit 21D acquires a speech voice signal (an example of a registered speech signal) associated with the same speaker information as the acquired speaker information among the speaker information of each of multiple speakers registered in the registered speaker database DB, and calculates a first similarity between the registered speech signal and a first speech section and a second similarity between the registered speech signal and a second speech section. The speaker change identification unit 21H detects a change from one speaker to another speaker based on a change between the first similarity and the second similarity. As a result, the authentication analysis device P1A in the second embodiment can distinguish between the speech voice signal of the speaker who owns the user-side communication terminal UP1 and the speech voice signals of other speakers based on the similarity with the speech voice signals registered in the registered speaker database DB. Therefore, the authentication analysis device P1A can more effectively prevent erroneous registration, in which the speech voice signal to be registered is mixed with the speech voice signal of another speaker and registered, or is overwritten.

[0170] Furthermore, as described above, the speaker change identifying unit 21H in the authentication analysis device P1A in embodiment 2 detects a change from one speaker to another when it determines that the similarity is not equal to or greater than the threshold. As a result, the authentication analysis device P1A in embodiment 2 can distinguish between the speech signal of the speaker who is the holder of the user-side communication terminal UP1 and the speech signal of another speaker, based on the similarity with the speech signal registered in the registered speaker database DB.

[0171] As described above, the authentication analysis device P1A in the second embodiment further includes an emotion identification unit 21E that identifies at least one emotion included in the audio signal, and an utterance interval detection unit 21A (an example of a deletion unit) that deletes utterance intervals including emotions based on the identification result by the emotion identification unit. The utterance interval detection unit 21A detects the speaker's first and second utterance intervals from the audio signal from which the utterance intervals including emotions have been deleted. This allows the authentication analysis device P1A in the second embodiment to extract only utterance audio signals that are more suitable for user authentication processing (utterance audio signals that do not include emotions in FIG. 14) based on the emotions included in the audio signal of the user US, and register (store) the extracted utterance audio signals in the registered speaker database DB.

[0172] As described above, the authentication analysis device P1A in the second embodiment further includes an emotion identification unit 21E that identifies at least one emotion included in the audio signal, and a communication unit 20 (an example of an input unit) that accepts an operation to delete an utterance section including an emotion based on the identification result by the emotion identification unit 21E. When the communication unit 20 accepts the operation to delete an utterance section, the speech section detection unit 21A deletes the utterance section including the emotion and detects a first utterance section and a second utterance section of the speaker from the audio signal from which the utterance section including the emotion has been deleted. As a result, the authentication analysis device P1A in the second embodiment allows the operator to select whether or not to register (store) an utterance audio signal including the emotion of the user US in the registered speaker database DB based on an operation of the OP.

[0173] As described above, the authentication analysis device P1A in the second embodiment further includes a speech section detection unit 21A (an example of a conversion unit) that converts the voice signal acquired by the communication unit 20 to a predetermined speech speed. The emotion identification unit 21E identifies emotions using the voice signal converted to the predetermined speech speed. As a result, the authentication analysis device P1A in the second embodiment can improve the accuracy of user authentication by converting the speech speed to the predetermined speech speed even when there is variation in the speech speed of speakers.

[0174] As a result, in the authentication analysis device P1A according to the second embodiment, the first utterance period and the second utterance period each include at least the same utterance period, which allows the authentication analysis device P1A according to the second embodiment to more accurately detect the timing of speaker switching.

[0175] As described above, in the authentication analysis device P1A according to the second embodiment, the speaker information is the telephone number of the user-side communication terminal UP1 (an example of a sound collection device) that collects the uttered voice. This allows the authentication analysis device P1A according to the second embodiment to more effectively prevent impersonation by other speakers.

[0176] (Background to the Third Embodiment) Patent Document 1 discloses a communication device that, when it is determined that the total acquisition time length of at least one or more voiceprint data corresponding to a telephone number in a telephone directory and the same telephone number as the acquired telephone number is longer than the time required for voiceprint matching, associates the acquired telephone number with the voiceprint data and stores the associated telephone number. However, the above-mentioned communication device collects the spoken voice (voiceprint data) based on the speaker's own operation, and when it is determined that the total acquisition time length of the voiceprint data is longer than the time required for voiceprint matching, it prompts the speaker to choose whether or not to register the voiceprint data. Therefore, when the total acquisition time length of the voiceprint data is shorter than the time required for voiceprint matching, the communication device has a problem in that it cannot store new voiceprint data if the speaker chooses not to register the voiceprint data.

[0177] Another method for registering voiceprint data is to extract only the speaker's voice from the voice of the conversation between the speaker and the operator, and store the extracted speaker's voice as voiceprint data. In this method of registering voiceprint data, the operator cannot know in real time the registration status of the speaker's voiceprint data that is currently acquired and stored, which can lead to a delay in responding to a customer during a call, or an inability to acquire and register sufficient voiceprint data during a call.

[0178] Therefore, in the following embodiment 3, an example of a speech voice registration device and a speech voice registration method will be described, which notify the current registration status of a speech voice signal in real time and support the operator in registering the speech voice signal. In the following description, the same components as those in embodiment 1 or 2 will be assigned the same reference numerals, and their description will be omitted.

[0179] (Embodiment 3) A block diagram showing an example of the internal configuration of the authentication analysis device P1B in the third embodiment will be described with reference to Fig. 17. Fig. 17 is a block diagram showing an example of the internal configuration of the authentication analysis device P1B in the third embodiment.

[0180] The authentication analysis device P1B in the third embodiment is an example of a computer, and generates example questions for the user US to acquire the missing number of sound types, speech length, etc., based on the missing number of sound types, speech length, etc., in the speech uttered by the user US. The authentication analysis device P1B generates the generated example question screen DP21 (see FIG. 20) or progress notification screens DP22A, DP22B, etc., and outputs them to the information display unit DP for display.

[0181] Processor 212 in embodiment 3 realizes the functions of speech section detection unit 21A, speech linking unit 21B, feature extraction unit 21C, similarity calculation unit 21D, emotion identification unit 21E, database registration unit 21F, and progress determination unit 21I.

[0182] The progress determination unit 21I determines the current progress based on the number of sound types, speech length, etc. that are missing in the speech uttered by the user US. If the progress determination unit 21I determines that the progress of the current registration process of the speech voice signal (i.e., the target registration data amount) is not equal to or greater than a threshold, it generates example questions for the user US to acquire the missing number of sound types, speech length, etc., based on the missing number of sound types, speech length, etc., in the multiple speech voice signals uttered by the user US.

[0183] The progress determination unit 21I generates progress notification screens DP22A, DP22B (see Figures 21 and 22) that notify the operator OP of the determined progress (in other words, the current registration status of the speech voice signal) in real time, and outputs them to the information display unit DP for display.

[0184] Furthermore, when the authentication analysis device P1B determines that the progress of the current registration process of the speech voice signal (i.e., the target registration data volume) is not equal to or greater than a threshold, it generates example questions for the user US to acquire the missing number of sound types, speech length (signal length), etc., based on the missing number of sound types, speech length (signal length), etc. in the speech uttered by the user US. The authentication analysis device P1B generates the generated example question screen DP21 (see FIG. 20) or progress notification screens DP22A, DP22B, etc., and outputs them to the information display unit DP for display.

[0185] The memory 221 in the third embodiment stores a plurality of example questions selected in accordance with the number of missing sound types, speech length, and the like.

[0186] Next, the progress determination process performed by the progress determination unit 21I in the third embodiment will be described with reference to Fig. 18. Fig. 18 is a diagram illustrating an example of the progress determination process in the third embodiment.

[0187] In the example shown in FIG. 18, each of the speech signals Us41, Us42, and Us43 is a speech signal of the user US, and indicates a speech signal to be registered in the registered speaker database DB.

[0188] The progress determination unit 21I determines the degree of progress to be "low" when a speech audio signal Us41 with 7 sound types and a speech length of 5 seconds is currently registered. Furthermore, the progress determination unit 21I determines the degree of progress to be "medium" when a speech audio signal Us42 with 12 sound types and a speech length of 10 seconds is currently registered. Furthermore, the progress determination unit 21I determines the degree of progress to be "high" when a speech audio signal Us43 with 15 sound types and a speech length of 13 seconds is currently registered. The progress determination unit 21I may evaluate the degree of progress as a percentage.

[0189] Here, the processor 212 may notify the operator OP of the degree of progress by audio output based on the degree of progress determined by the progress degree determination unit 21I. For example, the processor 212 outputs a predetermined notification sound only once when the degree of progress is "low", outputs the predetermined notification sound twice when the degree of progress is "medium", and outputs the predetermined notification sound three times when the degree of progress is "high". This allows the operator OP to know the current degree of progress based on the number of times the predetermined notification sound is sounded. Note that the predetermined notification sound may be a voice message.

[0190] Next, the operation procedure of the authentication analysis device in the third embodiment will be described with reference to Fig. 19. Fig. 19 is a flowchart showing an example of the operation procedure of the authentication analysis device P1B in the third embodiment.

[0191] The processor 212 determines whether the current progress is equal to or greater than a threshold value (St40).

[0192] When the processor 212 determines in the processing of step St40 that the current progress level is equal to or greater than the threshold value (St40, YES), it associates the speech voice signals linked by emotion with the acquired telephone number of the user side call terminal UP1, registers (stores) them in each emotion database of the registered speaker database DB (St41), and completes the registration processing of the speech voice signals (St42).

[0193] On the other hand, when it is determined in the processing of step St40 that the current progress degree is not equal to or greater than the threshold value (St40, NO), the processor 212 determines whether or not a control command requesting continuation of the registration processing has been received (St43).

[0194] If the processor 212 determines in the processing of step St43 that it has received a control command requesting continuation of the registration process based on the control command based on the operator OP operation transmitted from the operator-side call terminal OP1 (St43, YES), it selects at least one example question based on the currently lacking number of sound types of the speech voice signal, the speech length, etc. The processor 212 generates a question example screen DP21 (see FIG. 20) or a progress notification screen DP22A, DP22B (see FIGS. 21 and 22) including the selected example question, and outputs it to the information display unit DP for display (St44). The processor 212 proceeds to the processing of step St15.

[0195] On the other hand, if the processor 212 determines in the processing of step St43 that it has not received a control command requesting the continuation of the registration process based on the control command based on the operator OP operation transmitted from the operator-side call terminal OP1 (i.e., a control command requesting the cancellation or termination of the registration process has been received) (St43, NO), it cancels the registration process of the acquired speech voice signal to the registered speaker database DB (i.e., registration failed) (St45).

[0196] After the process of step St42 or step St45, the processor 212 generates a registration result screen (not shown) including the result of the registration process of the speech sound signal to the registered speaker database DB, and outputs it to the information display unit DP for display (St46).

[0197] As described above, the authentication analysis device P1B in the third embodiment can distinguish the speech voice signal of the user US by emotion and register (store) it in the registered speaker database DB. Therefore, even if the number of sound types and speech length contained in the speech voice signal vary depending on the emotion of the user US during a call, the authentication analysis device P1B can support the operator OP by presenting the operator OP with example questions for acquiring a speech voice signal including the number of sound types and speech length required for user authentication as the target data amount for registration.

[0198] Furthermore, the authentication analysis device P1B visualizes the determined progress to the operator OP in real time. Therefore, the operator OP can know the registration status of the speech voice signal in real time based on the answers to questions by the user US, and can more easily determine the timing to end the call with the user US, whether to ask the next question, etc.

[0199] Next, the example question screen DP21 will be described with reference to Fig. 20. Fig. 20 is a diagram illustrating an example of the example question screen DP21. Note that the example question screen DP21 shown in Fig. 20 is just an example, and is not limited to this. Also, the number of sound types graph IN212 shown in Fig. 20 is not essential and may be omitted.

[0200] The question example screen DP21 is generated and displayed in the information display section DP when the progress determination section 21I determines that the progress of the current registration process of the speech voice signal (that is, the target data amount for registration) is not equal to or greater than the threshold.

[0201] The progress determination unit 21I analyzes the number of sound types, speech length (signal length), etc. that are missing in the speech uttered by the user US, and based on the analysis results, selects at least one example question for acquiring the missing number of sound types, speech length, etc. as a speech audio signal to be used in user authentication processing. The progress determination unit 21I sets a priority for each of the at least one selected example question, generates an example question screen DP21 that includes at least one example question and a priority corresponding to the example question, and outputs it to the information display unit DP for display. Note that the priority information is not essential and may be omitted.

[0202] 20, the example question screen DP21 is generated to include four example questions and a sound type number graph IN212 that indicates the number of sound types for each sound in the speech uttered by the user US. From top to bottom, the example question screen DP21 includes the example question with priority 1, "Could you please tell me the address you would like to register?", the example question with priority 2, "Could you please tell me the phone number you would like to register?", the example question with priority 3, "Could you please tell me your date of birth?", and the example question with priority 4, "Could you please say 'ka-ki-ku-ke-ko'?"

[0203] This allows the operator OP to select one of the four example questions on the example question screen DP21 and ask it to the user US.

[0204] As a result, the authentication analysis device P1B in the third embodiment can more efficiently acquire the missing number of sound types, speech length, etc., and acquire a speech voice signal more suitable for user authentication. Also, the authentication analysis device P1B can support the operator OP so that the call between the user US and the operator OP does not drag on.

[0205] Next, progress notification screens generated by the progress degree determination unit 21I in the third embodiment will be described with reference to Fig. 21 and Fig. 22. Fig. 21 is a diagram illustrating an example of a progress notification screen DP22A for the user US. Fig. 22 is a diagram illustrating an example of a progress notification screen DP22B for the user US. Note that the progress notification screen DP22A shown in Fig. 21 and the progress notification screen DP22B shown in Fig. 22 are merely examples and are not limiting.

[0206] The progress notification screen DP22A is a screen for notifying the operator OP of the progress of the registration process of the speech voice signal acquired from the user-side call terminal UP1, and is generated by the processor 212 and displayed by the information display unit DP. When the processor 212 determines that the progress is less than a threshold, the progress notification screen DP22A is generated to further include an example question IN221 for acquiring the number of missing sound types and the speech length. The progress notification screen DP22A shown in Fig. 21 includes the example question IN221, a registration name input field IF21, registration progress information IN23, a voice signal graph IN24, and a speaker registration button BT21.

[0207] The example question IN221 includes, from top to bottom, the example question with priority 1, "Could you please tell me the address you would like to register?", and the example question with priority 2, "Could you please tell me the phone number you would like to register?".

[0208] When any one of the example questions displayed on the progress notification screen DP22A is selected (pressed) by the operator OP, the processor 212 may store this example question as an example question that has already been asked. When the processor 212 determines again that the progress level is less than the threshold, it may select an example question to present to the operator OP from each of the multiple example questions stored in the memory 22 other than the example question that has already been asked.

[0209] The registered name input field IF21 is an input field that can accept an operator's operation regarding a user name to be registered in the registered speaker database DB. Note that the registered name input field IF21 may be capable of inputting information about the user US.

[0210] The registration progress information IN23 includes a progress IN231 indicating the number of sound types of the currently registered speech voice signal relative to the number of sound types of the registration target data amount, and a progress IN232 indicating the utterance length of the currently registered utterance voice signal relative to the utterance length of the registration target data amount. Furthermore, the registration progress information IN23 may display the number of sound types of the currently registered utterance voice signal using numerical values, such as "Number of sound types: XX", and the utterance length, such as "Utterance length: XX".

[0211] The voice signal graph IN24 displays the speech voice signal of the detected speech section.

[0212] The speaker registration button BT21 accepts an operation by the operator OP to start or stop the registration process of the speech voice signal in the registered speaker database DB. When the speaker registration button BT21 accepts an operation by the operator OP to start the registration process of the speech voice signal in the registered speaker database DB, the speaker registration button BT21 registers the acquired speech voice signal in the registered speaker database DB. When the speaker registration button BT21 accepts an operation by the operator OP to stop the registration process of the speech voice signal in the registered speaker database DB, the speaker registration button BT21 stops the registration process of the speech voice signal in the registered speaker database DB.

[0213] The progress notification screen DP22B is generated by the processor 212 and displayed by the information display unit DP when the processor 212 determines that the degree of progress is less than the threshold value after the progress notification screen DP22A is displayed.

[0214] Example question IN222 is a newly generated example question, and includes, from top to bottom, the example question with priority 1, "May I have the telephone number you would like to register?", and the example question with priority 2, "May I have your date of birth?"

[0215] The registration progress information IN25 includes a progress IN251 indicating the number of sound types of the currently registered speech voice signal relative to the target registration data amount of the number of sound types, and a progress IN252 indicating the speech length of the currently registered speech voice signal relative to the target registration data amount of the speech length. Furthermore, the registration progress information IN25 may display the number of sound types of the currently registered speech voice signal using numerical values, such as "Number of sound types: ZZ" and the speech length, such as "Speech length: ZZ".

[0216] The voice signal graph IN26 displays the speech voice signal of the detected speech section.

[0217] As described above, the authentication analysis device P1B in embodiment 3 comprises a communication unit 20 (an example of an acquisition unit) that acquires an audio signal of a speaker's spoken voice and speaker information that can identify the speaker, a database registration unit 21F (an example of a registration unit) that associates the acquired audio signal with the speaker information and registers it in a registered speaker database DB (an example of a database), a progress determination unit 21I that repeatedly determines the progress (an example of a registration progress) of the audio signal to be registered in the registered speaker database DB against the registration target data amount (an example of a registration target amount) of the audio signal to be registered in the registered speaker database DB, and a processor 212 (an example of a notification unit) that notifies the determined progress.

[0218] As a result, the authentication analysis device P1B in the third embodiment visualizes the determined progress to the operator OP in real time. Therefore, the operator OP can know the registration status of the speech voice signal in real time based on the answers to the questions by the user US, and can more easily determine the timing to end the call with the user US, whether to ask the next question, etc.

[0219] Furthermore, as described above, the progress determination unit 21I in the authentication analysis device P1B in embodiment 3 repeatedly determines the progress of the voice signal registered in the registered speaker database DB and the number of sound types included in the registered voice signal. The processor 212 notifies the progress and the number of sound types. As a result, the authentication analysis device P1B in embodiment 3 visualizes the determined progress to the operator OP in real time. Therefore, the operator OP can know the registration status (progress and number of sound types) of the speech voice signal in real time based on the answers to questions by the user US, and can more easily determine the timing to end the call with the user US, whether to ask the next question, etc.

[0220] Furthermore, as described above, the progress determination unit 21I in the authentication analysis device P1B in the third embodiment determines the number of sound types for each sound included in the voice signal registered in the registered speaker database DB (for example, the sound type number graph IN212 shown in FIG. 20). The processor 212 notifies the number of sound types for each sound. This allows the authentication analysis device P1B in the third embodiment to visualize the number of sound types for each sound, which is the registration status of the speech voice signal, and notify the operator OP.

[0221] Furthermore, as described above, the progress determination unit 21I in the authentication analysis device P1B in embodiment 3 repeatedly determines the progress of the voice signal registered in the registered speaker database DB and the signal length of the registered voice signal. The processor 212 notifies the progress and the signal length. As a result, the authentication analysis device P1B in embodiment 3 visualizes the determined progress to the operator OP in real time. Therefore, based on the answers to questions from the user US, the operator OP can know the registration status of the speech voice signal (progress and speech length (signal length)) in real time, and can more easily determine the timing to end the call with the user US, whether to ask the next question, etc.

[0222] As described above, the authentication analysis device P1B in the third embodiment further includes a processor 212 (an example of a selection unit) that selects at least one question to the user US based on the progress level. The processor 212 generates and outputs progress notification screens DP22A and DP22B (see FIGS. 21 and 22) that include the selected question and the progress level. This allows the authentication analysis device P1B in the third embodiment to acquire a speech signal that is more suitable for user authentication based on the progress level.

[0223] As described above, the progress determination unit 21I in the authentication analysis device P1B in the third embodiment repeatedly determines the progress of the voice signal registered in the registered speaker database DB and the number of sound types contained in the registered voice signal. The processor 212 selects questions based on the number of sound types in the target registration data amount and the number of sound types contained in the registered voice signal. This allows the authentication analysis device P1B in the third embodiment to more efficiently acquire the missing number of sound types and acquire a speech voice signal more suitable for user authentication.

[0224] Furthermore, as described above, the processor 212 in the authentication analysis device P1B in embodiment 3 selects a plurality of questions based on the number of sound types in the target data amount for registration and the number of sound types included in the voice signal to be registered, and determines the priority of each of the selected questions. The processor 212 generates and outputs progress notification screens DP22A and DP22B (see FIGS. 21 and 22) that include each of the selected questions, the priority of the question, and the progress. As a result, the authentication analysis device P1B in embodiment 3 more efficiently acquires the missing number of sound types and visualizes to the operator OP, in order of priority, example questions from which a speech voice signal more suitable for user authentication can be acquired.

[0225] Furthermore, as described above, the progress determination unit 21I in the authentication analysis device P1B in embodiment 3 repeatedly determines the progress of the voice signal registered in the registered speaker database DB and the signal length included in the registered voice signal. The processor 212 selects a question based on the signal length of the target data amount to be registered and the signal length included in the voice signal to be registered. As a result, the authentication analysis device P1B in embodiment 3 selects a question based on the signal length (utterance length) of the target data amount to be registered and the signal length (utterance length) included in the voice signal to be registered. As a result, the authentication analysis device P1B in embodiment 3 can more efficiently acquire the missing utterance length and acquire a spoken voice signal more suitable for user authentication.

[0226] Furthermore, as described above, the processor 212 in the authentication analysis device P1B in the third embodiment generates and outputs progress notification screens DP22A and DP22B (see FIGS. 21 and 22) including the progress degree. As a result, the authentication analysis device P1B in the third embodiment visualizes the determined progress degree to the operator OP in real time. Therefore, the operator OP can know the registration status of the speech voice signal in real time based on the answers to the questions by the user US, and can more easily determine the timing of ending the call with the user US, whether to ask the next question, etc.

[0227] Furthermore, as described above, the processor 212 in the authentication analysis device P1B in the third embodiment generates and outputs an audio notification signal that notifies the progress by voice. This allows the authentication analysis device P1B in the third embodiment to notify the determined progress to the operator OP in real time. Therefore, the operator OP can know the registration status of the speech voice signal in real time based on the answers to questions by the user US, and can more easily determine the timing to end the call with the user US, whether to ask the next question, etc.

[0228] Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, alterations, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure. Furthermore, the components of the various embodiments described above may be combined in any manner without departing from the spirit of the invention.

[0229] This application is based on a Japanese patent application (Patent Application No. 2021-197228) filed on December 3, 2021, the contents of which are incorporated by reference into this application. [Industrial Applicability]

[0230] The present disclosure is useful as a voice registration device and a voice registration method that notify the current registration status of a speech voice signal in real time and support the registration work of a speech voice signal performed by an operator. [Explanation of symbols]

[0231] 20 Communications Department 210,211,212 processors 21A Speech activity detector 21B Speech connector 21C Feature Extraction Unit 21D Similarity calculation part 21E Emotion Identification Section 21F Database Registration Department 21G, 21I Progress Determination Section 22 Memory 22A ROM 22B RAM 23 Display I / F 100 Voice Authentication System DB Registered speaker database DB1 The first emotion database DB2 Second Emotion Database DBN Nth Emotion Database DP information display section DP22A, DP22B progress notification screen OP1 Operator side call terminal P1 Authentication Analysis Device SC authentication result screen US users UP1 User side call terminal

Claims

1. an acquisition unit that acquires a voice signal of a speaker's speech and speaker information that can identify the speaker; a registration unit that associates the acquired voice signal with the speaker information and registers them in a database; a progress determination unit that repeatedly determines the progress of registration of audio signals to be registered in the database, the signal length of the registered audio signals, the number of sound types included in the registered audio signals, and the number of sound types for each sound, with respect to a registration target amount of audio signals to be registered in the database; a selection unit that selects a plurality of questions to the speaker based on the degree of registration progress, the target number of sound types to be registered, and the number of sound types included in the voice signal to be registered; a notification unit that generates and outputs a progress notification screen that notifies the user of the registration progress level, the selection unit determines a priority order for each of the selected questions; the notification unit generates and outputs the progress notification screen, which includes each of the selected questions, the priority of the question, the registration progress, the number of sound types included in the audio signal, and the number of sound types for each of the sounds. Voice registration device.

2. the progress determination unit repeatedly determines the registration progress of the audio signal registered in the database and a signal length included in the audio signal to be registered; the selection unit selects the question based on the signal length of the registration target amount and the signal length included in the voice signal to be registered.

2. The voice registration device according to claim 1.

3. the notification unit generates and outputs a voice notification signal that notifies the registration progress by voice.

2. The voice registration device according to claim 1.

4. 1. A method of voice enrollment performed by one or more computers, comprising: Acquiring a voice signal of a speaker's speech and speaker information that can identify the speaker; registering the acquired voice signal and the speaker information in a database in association with each other; repeatedly determining the progress of registration of the audio signals to be registered in the database, the signal length of the registered audio signals, the number of sound types included in the audio signals to be registered in the database, and the number of sound types for each sound, with respect to a target registration amount of the audio signals to be registered in the database; selecting a plurality of questions to the speaker based on the degree of registration progress, the target number of sound types to be registered, and the number of sound types included in the voice signal to be registered; determining a priority for each of the selected questions; generating and outputting a progress notification screen that includes each of the selected questions, the priority of the questions, the registration progress level, the number of sound types included in the audio signal, and the number of sound types for each sound, and that notifies the registration progress level; How to register voice.

Images