Speaker recognition device and operating procedure therefor

Abstract

One purpose of the present disclosure is to ensure the performance of speaker recognition, even if a speech input device for a spoken voice differs from a speech input device for a voice spoken in a speaker registration process.

Description

Technical field

[0001] The present disclosure relates to a speaker recognition device and, in particular, to a speaker recognition device for recognizing a speaker via a plurality of speech input devices. State of the art

[0002] Speaker recognition refers to a service that uses speech recognition technology to identify and distinguish specific speakers. Speaker recognition services can be used in a variety of applications, including security, education, voice commands, and automation systems.

[0003] The speaker recognition service analyzes the characteristics of the voice uttered by a speaker, stores a unique speech pattern of each speaker, and identifies the speaker based on the stored speech pattern.

[0004] The speaker recognition service requires a speaker registration process to register a speaker's voice. During speaker registration, keywords are provided to increase the accuracy of speech recognition.

[0005] The speaker utters a keyword and a speech recognition device such as a mobile device or a television identifies the speaker by recognizing the uttered keyword.

[0006] Speaker-dependent speaker recognition includes a speaker registration process that registers a speaker based on the voice uttered by a user by speaking a presented sentence, and a speaker identification process that determines, based on the uttered voice, whether the speaker is registered while using the speech recognition service.

[0007] For speech recognition of a speaker, there are various speech input devices (remote control device, remote microphone, headset, mobile device and the like).

[0008] However, the acoustic properties of a microphone differ for each speech input device, and if an input device for a voice uttered in the speaker registration process and an input device for the voice uttered in the speaker identification process are different, speaker recognition may not be achieved well due to differing microphone performance. Revelation Technical Problem

[0009] One purpose of the present disclosure is to ensure the performance of a speaker recognition system, even if an input device for a spoken voice differs from an input device for the voice spoken in the speaker identification process.

[0010] One purpose of the present disclosure is to improve the performance of speaker recognition by storing the features of a speech input device that is not registered in the speaker identification process and voices received via the speech input device. Technical solution

[0011] According to one embodiment of the present disclosure, a speaker recognition device comprises a communication interface configured to communicate with an electronic device, a memory configured to store a speaker profile comprising a first set of speech features including speech features of a first speech signal and a first speech input channel corresponding to the first speech signal, and a processor configured to receive from an electronic device a second speech signal corresponding to a reception of a speech command uttered by a speaker, and information about a second speech input channel indicating a device receiving the speech command, and determining a similarity between the first set of speech features and a second set of speech features including speech features of the second speech signal based thereon.that the second speech input channel is not the first speech input channel stored in memory, and sending a notification to the electronic device indicating that the speaker has been identified based on the fact that the received similarity is equal to or greater than a first similarity.

[0012] The processor can compare a first embedding vector, which specifies the first language feature set, with a second embedding vector, which specifies the second language feature set, in order to obtain the similarity.

[0013] Based on the fact that the similarity is less than the first similarity and equal to or greater than a second similarity that is less than the first similarity, the processor can be configured to send a notification to the electronic device to verify that the speaker matches a pre-registered speaker.

[0014] Based on the fact that the similarity is less than the first similarity and equal to or greater than a second similarity that is less than the first similarity, the processor can be configured to send a notification to the electronic device to register the second speech input channel.

[0015] The processor can be configured to receive additional speech features of an additional speech signal corresponding to an additional speech command received via the second speech input channel from the electronic device and to update the information about the second speech input channel and the additional speech features in the speaker profile.

[0016] Based on the fact that the similarity is less than the first similarity and equal to or greater than a second similarity that is less than the first similarity, the processor may be configured to send a notification to the electronic device via the first speech input channel, prompting an additional utterance of a keyword.

[0017] Based on the fact that the similarity is less than the second similarity, the processor can be configured to send a notification to the electronic device to register a new speaker.

[0018] Both the first and second voice input channels can each include a remote control device, the electronic device and a mobile device, each including a microphone.

[0019] According to one embodiment of the present disclosure, an operating method of a speaker recognition device comprises storing a speaker profile including a first speech feature set containing speech features of a first speech signal and a first speech input channel corresponding to the first speech signal, receiving a second speech signal corresponding to a speech command uttered by a speaker and information about a second speech input channel indicating that a device is receiving the speech command from an electronic device, determining a similarity between the first speech feature set and a second speech feature set including speech features of the second speech signal, based on the fact that the second speech input channel is not the first speech input channel stored in memory, and sending a notification indicating that the speaker has been identified.to the electronic device based on the fact that the obtained similarity is equal to or greater than a first similarity.

[0020] Maintaining similarity may involve comparing a first embedding vector, which specifies the first set of language features, with a second embedding vector, which specifies the second set of language features, in order to preserve the similarity.

[0021] The procedure may further include, based on the fact that the similarity is less than the first similarity and equal to or greater than a second similarity that is less than the first similarity, sending a notification to the electronic device to verify that the speaker matches a pre-registered speaker.

[0022] The procedure may further include, based on the fact that the similarity is less than the first similarity and equal to or greater than a second similarity that is less than the first similarity, sending a notification to the electronic device to register the second voice input channel.

[0023] The procedure may further include receiving additional speech features of an additional speech signal corresponding to an additional speech command received via the second speech input channel from the electronic device and updating the information about the second speech input channel and the additional speech features in the speaker profile.

[0024] The procedure may further include, based on the fact that the similarity is less than the first similarity and equal to or greater than a second similarity that is less than the first similarity, sending a notification to the electronic device via the first speech input channel, prompting an additional utterance of a keyword.

[0025] The procedure may also include sending a notification to the electronic device to register a new speaker, based on the fact that the similarity is less than the second similarity. Beneficial effects

[0026] According to one embodiment of the present disclosure, speaker recognition can be performed normally even if a speaker expresses his voice via any channel of a plurality of speech input channels.

[0027] That is, according to one embodiment of the present disclosure, the performance of speaker recognition can be improved even if a speaker expresses his voice via different speech input channels. Description of the drawings Fig. Figure 1 is a block diagram representing a configuration of a display device according to an embodiment of the present disclosure. Fig. Figure 2 is a block diagram representing a remote control device according to an embodiment of the present disclosure. Fig. Figure 3 is a view that represents an actual configuration of a remote control device according to an embodiment of the present disclosure. Fig. Figure 4 is a view of a use of a remote control device according to an embodiment of the present disclosure. Fig. Figure 5 shows a server for artificial intelligence (AI) according to an embodiment of the present disclosure. Fig. Figure 6 is a diagram illustrating a configuration of a speaker recognition system according to an embodiment of the present disclosure. Fig. Figure 7 is a contact diagram to explain an operating procedure of a speaker recognition system according to an embodiment of the present disclosure. Fig. 8A is a diagram illustrating a speaker registration image according to an embodiment of the present disclosure, Fig. 8B is a diagram explaining keywords provided by the speaker registration image, and Fig. 8C is a diagram illustrating multiple speech input channels. Fig. Figure 9 is a diagram illustrating a case in which a speaker who is speaking is identified via the same speech input channel according to an embodiment of the present disclosure. Fig. Figure 10 is a diagram illustrating an example in which a speaker registration notification is issued to initiate speaker registration with another account according to an embodiment of the present disclosure. Fig. Figures 11A to 11C are diagrams illustrating a process for updating an existing speaker profile by adding information about a new speech input channel for the same speaker according to an embodiment of the present disclosure. Best mode

[0028] In the following, embodiments relating to the present disclosure are described in detail with reference to the drawings. The suffixes "module" and "unit" for components used in the following description are assigned or combined for the sake of simplicity in the specification and have no inherent meaning or role.

[0029] A display device according to one embodiment of the present disclosure is, for example, an intelligent display device that adds a computer support function to a broadcast reception function and may have a user-friendly interface such as a writing input device, a touchscreen, or a spatial remote control device, since an internet function is added while simultaneously fulfilling the broadcast reception function. Then, with the support of a wired or wireless internet function, it is possible to perform email, web browsing, banking, or gaming functions when accessing the internet and computers. Standardized general-purpose operating systems can be used to perform these various functions.

[0030] Since various applications can be freely added to or removed from a general-purpose operating system kernel, a display device described here can accordingly perform various user-friendly functions. More specifically, the display device can be a network television, a hybrid broadcast broadband television (HBBTV), a smart TV, an LED television, an organic light-emitting diode (OLED) television, etc., and in some cases, it can be a smartphone.

[0031] Fig. Figure 1 is a block diagram representing a configuration of a display device according to an embodiment of the present disclosure.

[0032] With reference to Fig. 1 can include a display device 100, a broadcast receiver 130, an external device interface 135, a memory 140, a user interface 150, a control 170, a wireless communication interface 173, a display 180, an audio output interface 185 and a power supply 190.

[0033] The broadcast receiver 130 can include a tuner 131, a demodulator 132 and a network interface 133.

[0034] The tuner 131 can select a specific broadcast channel according to a channel selection command. The tuner 131 can receive broadcast signals for the selected specific broadcast channel.

[0035] The demodulator 132 can split the received broadcast signals into video signals, audio signals and broadcast program-related data signals, and can restore the split video signals, audio signals and data signals into an output-ready form.

[0036] The external device interface 135 can receive an application or application list in an adjacent external device and send the application or application list to the controller 170 or the memory 140.

[0037] The external device interface 135 can provide a connection path between the display device 100 and the external device. The external device interface 135 can receive at least one image or audio output from the external device, which is connected wirelessly or via a wired connection to the display device 100, and can send the image and / or sound to the controller 170. The external device interface 135 can include multiple external input connectors. These multiple external input connectors can include an RGB connector, at least one HDMI (High Definition Multimedia Interface) connector, and one component connector.

[0038] A video signal from an external device, input via external device interface 135, can be displayed on the screen 180. A voice signal from an external device, input via external device interface 135, can be output via audio output interface 185.

[0039] An external device that can be connected to the external device interface 135 could be a side-by-side device, a Blu-ray player, a DVD player, a game console, a soundbar, a smartphone, a PC, a USB storage device, or a home theater system, but this is only an example.

[0040] The network interface 133 can provide an interface for connecting the display device 100 to a wired / wireless network, including an internet network. The network interface 133 can send or receive data to or from another user or electronic device over a network that is accessed or another network that is connected to the network that is accessed.

[0041] In addition, some content data stored in the Display Device 100 can be sent to a user or electronic device selected from other users or other electronic devices pre-registered in the Display Device 100.

[0042] Network interface 133 can access a predefined web page via a network that is being accessed, or via another network connected to the network that is being accessed. That is, network interface 133 can send data to or receive data from a corresponding server by accessing a predefined web page over the network.

[0043] Network interface 133 can receive content or data provided by a content provider or network operator. This means that network interface 133 can receive content such as movies, advertisements, games, VODs, and broadcast signals provided by the content provider or network operator, as well as related information about the network.

[0044] In addition, network interface 133 can receive firmware update information and update files provided by the network operator and send data to the Internet or the content provider or the network operator.

[0045] Network interface 133 can select and receive a desired application from publicly available applications over the network.

[0046] The memory 140 can store signal-processed image, speech or data signals stored by a program in order to process and control each signal in the controller 170.

[0047] In addition, the memory 140 can perform a function for temporarily storing image, voice or data signals output by the external device interface 135 or the network interface 133, and can store information about a given image via a channel memory function.

[0048] Memory 140 can store an application or a list of applications entered by the external device interface 135 or the network interface 133.

[0049] The display device 100 can play content files (e.g., video files, still image files, music files, document files, application files, etc.) stored in memory 140 and make the content files available to a user.

[0050] The user input interface 150 can send signals entered by a user to the controller 170, or it can send signals from the controller 170 to a user. For example, the user input interface 150 can receive or process control signals such as power on / off, channel selection, and screen settings from a remote control device 200, or it can send control signals from the controller 170 to the remote control device 200 using various communication methods such as Bluetooth, ultra-wideband (WB), ZigBee, radio frequency (RF), and infrared (IR) communication methods.

[0051] In addition, the user input interface can send 150 control signals to the controller 170, which are entered by local buttons (not shown) such as an on / off button, a channel button, a volume button and a settings button.

[0052] Image signals processed by the controller 170 can be input to the display 180 and shown as corresponding images. Furthermore, image signals processed by the controller 170 can be input to an external output device via the external device interface 135.

[0053] Speech signals processed by the controller 170 can be output to the audio output interface 185. Furthermore, speech signals processed by the controller 170 can be input into the external output device via the external device interface 135.

[0054] Furthermore, the control unit 170 can control the overall operation of the display device 100.

[0055] In addition, the controller 170 can control the display device 100 by means of a user command or internal program input via the user input interface 150 and access the network to download a desired application or application list to the display device 100.

[0056] The controller 170 can output user-selected channel information along with the processed image or speech signals via the display 180 or the audio output interface 185.

[0057] In addition, the controller 170 can output image signals or voice signals from an external device such as a camera or camcorder, which are entered via the external device interface 135, via the display 180 or the audio output interface 185 according to an image playback command received from an external device via the user input interface 150.

[0058] Furthermore, the controller 170 can control the display 180 to show images, and can control the display 180 to show broadcast images entered via the tuner 131, external input images entered via the external device interface 135, images entered via the network interface, or images stored in memory 140. In this case, an image displayed on the display 180 can be a still image or a video, and can also be a 2D image or a 3D image.

[0059] In addition, the control unit 170 can play back stored content, received broadcast content and externally entered input content in the display unit 100, and the content can be in various formats, such as broadcast images, external input images, audio files, still images, accessed web screens and document files.

[0060] Furthermore, the wireless communication module 173 can perform wired or wireless communication with an external device. The wireless communication module 173 can perform short-range communication with an external device. For this purpose, the wireless communication interface 173 can support short-range communication by using at least one of the following technologies: Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra-Wideband (UWB), ZigBee, Near Field Communication (NFC), Wireless Fidelity (Wi-Fi), Wi-Fi Direct, and Wireless Universal Serial Bus (USB).The wireless communication module 173 can support wireless communication between the display device 100 and a wireless communication system, between the display device 100 and another display device 100, or between networks, including between the display device 100 and another display device 100 (or an external server), via wireless networks. The wireless networks can be wireless personal area networks.

[0061] The other display device 100 can be a mobile device such as a wearable device (e.g., a smartwatch, smart glasses, and a head-mounted display; HMD) or a smartphone that can exchange data with (or interact with) the display device 100. The wireless communication module 173 can detect (or recognize) a wearable device capable of communicating around the display device 100.

[0062] If the detected portable device is also authenticated for communication with the display device 100, the controller 170 can send at least some of the data processed in the display device 100 to the portable device via the wireless communication interface 173. Therefore, a user of the portable device can use the data processed by the display device 100 via the portable device.

[0063] The display 180 can convert image signals, data signals or on-screen display (OSD) signals processed in the controller 170, or image signals or data signals received in the external device interface 135, into R, G and B signals for generating driving signals.

[0064] Furthermore, the in Fig. The display device 100 shown is only one embodiment of the present disclosure and therefore some of the components shown may be integrated, added or omitted according to the specification of the display device 100 actually implemented.

[0065] This means that, if required, two or more components can be integrated into one component, or one component can be divided and configured into two or more components. Furthermore, one function performed by each block is to describe an embodiment of the present disclosure, and its specific operation or device does not limit the scope of the present disclosure.

[0066] According to another embodiment of the present disclosure, the display device 100, in contrast to Fig. 1. Receive and play back images via the network interface 133 or the external device interface 135 without using the tuner 131 and the demodulator 132.

[0067] For example, the display device 100 can be divided into an image processing device such as an auxiliary device for receiving broadcast signals or content according to various network services and a content playback device for playing back content that is input by the image processing device.

[0068] In this case, an operating method of a display device according to an embodiment of the present disclosure described below can be derived from one of the methods referred to in Fig. 1 described display devices, an image processing device such as the separate auxiliary device and a content playback device including the display 180 and the audio output interface 185.

[0069] A remote control device according to an embodiment of the present disclosure is described with reference to the Fig. 2 and Fig. 3 described.

[0070] Fig. Figure 2 is a block diagram representing a remote control device according to an embodiment of the present disclosure, and Fig. Figure 3 is a view that represents an actual configuration of a remote control device according to an embodiment of the present disclosure.

[0071] First, with reference to Fig. 2 comprising a remote control device 200, a fingerprint reader 210, a wireless communication module 220, a user input interface 230, a sensor module 240, an output interface 250, a power supply 260, a memory 270, a control 280 and a speech recorder 290.

[0072] With reference to Fig. 2 The wireless communication interface 220 sends / receives signals to / from any display device according to the above-mentioned embodiments of this disclosure.

[0073] The remote control device 200 can include a radio frequency (RF) module 221, which can send or receive signals to or from the display device 100 according to an RF communication standard, and an infrared (IR) module 223, which can send or receive signals to or from the display device 100 according to an IR communication standard. Furthermore, the remote control device 200 can include a Bluetooth module 225, which can send or receive signals to or from the display device 100 according to a Bluetooth communication standard. Additionally, the remote control device 200 can include an NFC module 227, which can send or receive signals to or from the display device 100 according to an NFC communication standard, and a wireless LAN (WLAN) module 229, which can send or receive signals to or from the display device 100 according to a WLAN communication standard.

[0074] In addition, the remote control device 200 can send signals containing information about the movement of the remote control device 200 to the display device 100 via the wireless communication module 220.

[0075] In addition, the remote control device 200 can receive signals sent by the display device 100 via the RF module 221 and, if necessary, send a command to the display device 100 to switch on / off, change channel and change volume via the IR module 223.

[0076] The user input interface 230 can be configured with a keyboard, a button, a touchpad, or a touchscreen. A user can operate the user input interface 230 to enter a command relating to the display device 100 into the remote control device 200. If the user input interface 230 includes a hard key, a user can enter a command relating to the display device 100 into the remote control device 200 by pressing the hard key. This is indicated by Fig. 3 described.

[0077] With reference to Fig. 3. The remote control device 200 can include a plurality of buttons. The plurality of buttons can include a fingerprint recognition button 212, a power button 231, a home screen button 232, a live button 233, an external input button 234, a volume control button 235, a voice recognition button 236, a channel change button 237, an OK button 238, and a back button 239.

[0078] The fingerprint recognition button 212 can be a button for recognizing a user's fingerprint. According to one embodiment of the present disclosure, the fingerprint recognition button 212 can perform a printing operation and receive a printing operation and a fingerprint recognition operation.

[0079] The on / off button 231 can be a button for switching the power supply of the display device 100 on / off.

[0080] The home screen button 232 can be a button to switch to the home screen of the display device 100.

[0081] The Live button 233 can be a button for viewing live broadcasts.

[0082] The external input key 234 can be a key for receiving an external input connected to the display device 100.

[0083] The volume control button 235 can be a button for controlling a volume output from the display device 100.

[0084] The speech recognition key 236 can be a key for receiving the user's voice and for recognizing the received voice.

[0085] The channel change button 237 can be a button for receiving broadcast signals from a specific broadcast channel.

[0086] The OK button 238 can be a button to select a specific function, and the Back button 239 can be a button to return to a previous screen.

[0087] Fig. 2 will be described again.

[0088] If the user input interface 230 includes a touchscreen, a user can touch a softkey button on the touchscreen to input a command relating to the display device 100 into the remote control device 200. Furthermore, the user input interface 230 can include various types of input interfaces that can be operated by a user, such as a scroll button and a rotary knob button, and this embodiment does not limit the scope of the present disclosure.

[0089] The sensor module 240 can include a gyroscope 241 or an accelerometer 243. The gyroscope 241 can detect information about the movement of the remote control device 200.

[0090] For example, the gyro sensor 241 can acquire information about the operation of the remote control device 200 based on the x, y, and z axes, and the accelerometer 243 can acquire information about the speed of movement of the remote control device 200. Furthermore, the remote control device 200 can also include a distance sensor that detects a distance relative to the display 180 of the display device 100.

[0091] The output interface 250 can output image or voice signals in response to the actuation of the user input interface 235, or it can output image or voice signals corresponding to the signals transmitted by the display device 100.

[0092] A user can use the output interface 250 to determine whether the user input interface 235 is being operated or the display device 100 is being controlled.

[0093] For example, the output interface 250 can include an LED module 251 for blinking, a vibration module 253 for generating vibration, a sound output module 255 for outputting sound, or a display module 257 for outputting an image when the user input interface 235 is manipulated or signals are sent / received to / from the display device 100 via the wireless communication interface 225.

[0094] In addition, the power supply 260 provides power to the remote control device 200 and stops the power supply if the remote control device 200 does not move for a predetermined period of time, thus reducing power waste.

[0095] The power supply 260 can resume power supply when a predetermined button is pressed, which is provided on the remote control device 200.

[0096] The memory 270 can store various types of programs and application data required to control or operate the remote control device 200.

[0097] When the remote control device 200 wirelessly transmits / receives signals via the display device 100 and the RF module 221, the remote control device 200 and the display device 100 transmit / receive signals over a specified frequency band.

[0098] The control unit 280 of the remote control device 200 can store and refer to information in memory 270 about a frequency band for sending / receiving signals to / from the display device 100 paired with the remote control device 200.

[0099] The controller 280 manages general matters related to the control of the remote control device 200. The controller 280 can send a signal corresponding to a predetermined key press of the user input interface 235, or a signal corresponding to the movement of the remote control device 200 detected by the sensor module 240, to the display device 100 via the wireless communication module 225.

[0100] In addition, the speech recorder 290 of the remote control device 200 can record speech.

[0101] The speech recorder 290 can include at least one microphone and capture the voice via the microphone.

[0102] Next will Fig. 4 described.

[0103] Fig. Figure 4 is a view that provides an example of the use of a remote control device according to an embodiment of the present disclosure.

[0104] Fig. Figure 4(a) shows that a pointer 205, corresponding to the remote control device 200, is displayed on the display 180.

[0105] A user can move or rotate the remote control device 200 vertically or horizontally. The pointer 205 displayed on the display 180 of the display device 100 corresponds to a movement of the remote control device 200. Since the corresponding pointer 205 is moved and displayed according to a movement in 3D space, as shown in the drawing, the remote control device 200 can be described as a spatial remote control device.

[0106] Fig. 4(b) shows that when a user moves the remote control device 200, the pointer 205 shown on the display 180 of the display device 100 is moved to the left in accordance with the movement of the remote control device 200.

[0107] Information about a movement of the remote control device 200 detected by a sensor of the remote control device 200 is sent to the display device 100. The display device 100 can calculate the coordinates of the pointer 205 from the information about the movement of the remote control device 200. The display device 100 can then display the pointer 205 according to the calculated coordinates.

[0108] Fig. Figure 4(c) shows that a user moves the remote control 200 away from the display 180 while a specific button on the remote control device 200 is pressed. This allows a selected area on the display 180, corresponding to the pointer 205, to be zoomed in and enlarged.

[0109] On the other hand, if a user moves the remote control device 200 close to the display 180, a selection area in the display 180, corresponding to the pointer 205, can be zoomed out and displayed in a reduced size.

[0110] On the other hand, if the remote control device 200 is moved away from the display 180, a selection area can be reduced in size, and if the remote control device 200 is moved closer to the display 180, a selection area can be zoomed in.

[0111] Furthermore, if a specific button on the remote control device 200 is pressed, detection of vertical or horizontal movement can be excluded. That is, if the remote control device 200 is moved away from or closer to the display 180, upward, downward, left, or right movement may not be detected, and only forward and backward movement may be detected. While a specific button on the remote control device 200 is not pressed, only the pointer 205 will move up, down, left, or right in accordance with the movement of the remote control device 200.

[0112] Furthermore, the speed or direction of movement of the pointer 205 can correspond to the speed or direction of movement of the remote control device 200.

[0113] Furthermore, in this description, a pointer refers to an object that appears on display 180 in response to an action on the remote control device 200. Therefore, in addition to the arrow shape shown in the drawing as pointer 205, various other object shapes are possible. The concept above includes, for example, a point, a cursor, a prompt, and a thick outline. Pointer 205 can then be displayed on display 180 corresponding to a point on a horizontal or vertical axis, and can also be displayed corresponding to multiple points, such as a line or an area.

[0114] Fig. Figure 5 shows a server for artificial intelligence (AI) according to an embodiment of the present disclosure.

[0115] With reference to Fig. 5. An AI server 500 can refer to a device that learns an artificial neural network using a machine learning algorithm or uses a learned artificial neural network.

[0116] The AI ​​Server 500 can comprise multiple servers to perform distributed processing, or it can be defined as a 5G network.

[0117] The AI ​​Server 500 can be included as a partial configuration of the AI ​​Device 100 and can jointly perform at least some of the AI ​​processing.

[0118] The AI ​​server 500 can include a communication unit 510, a memory 530, a learning processor 540 and a processor 560.

[0119] The communication unit 510 sends and receives data to and from an external device such as the display device 100.

[0120] The memory 530 can include a model memory unit 531.

[0121] The model storage unit 531 can store a learning or learned model (or an artificial neural network 531a) via the learning processor 540.

[0122] The learning processor 540 can train the artificial neural network 531a using the training data. The learning model can be used in a state where it is mounted on the AI ​​server 500 of the artificial neural network, or it can be used in a state where it is mounted on an external device such as the display device 100.

[0123] The learning model can be implemented in hardware, software, or a combination of both. If all or part of the learning model is implemented in software, one or more instructions that constitute the learning model can be stored in memory 530.

[0124] The 560 processor can derive the result value for new input data using the learning model and can generate a response or control command based on the derived result value.

[0125] Fig. Figure 6 is a diagram illustrating a configuration of a speaker recognition system according to an embodiment of the present disclosure.

[0126] With reference to Fig. 6 can include a speaker recognition system 60, the display device 100, a gateway 610, a speech-to-text (STT) server 620, a natural language processing (NLP) server 630, and a speaker recognition server 600.

[0127] The components that make up the speaker recognition system 60 can communicate wirelessly with each other. This wireless communication can be internet communication.

[0128] The speaker recognition server 600 can be described as a speaker recognition device.

[0129] The display device 100 can receive an initial voice command uttered by a speaker.

[0130] The display device 100 can send an initial speech signal and initial speech input channel information corresponding to the initial speech command to the speaker recognition server 600 via the network interface 133.

[0131] The processor 560 of the speaker recognition server 600 can receive an initial speech feature set that specifies features of the initial speech signal and store a speaker profile including the initial speech feature set and the initial speech input channel information in memory 530 or a speaker database 601.

[0132] The processor 560 of the speaker recognition server 600 can send a speaker registration notification to the display device 100 via the communication unit 510.

[0133] The control unit 170 of the display device 100 can output the speaker registration notification received from the speaker recognition server 600.

[0134] The control unit 170 of the display device 100 can receive a second voice command uttered by the speaker after a speaker registration process.

[0135] The control unit 170 of the display device 100 can send a second speech signal and second speech input channel information corresponding to the second speech command to the speaker recognition server 600 via the network interface 133.

[0136] The processor 560 of the speaker recognition server 600 can determine whether the first speech input channel and the second speech input channel are the same.

[0137] If it is determined that the first speech input channel and the second speech input channel are the same, the processor 560 of the speaker recognition server 600 can perform a speaker identification process.

[0138] If it is determined that the first speech input channel and the second speech input channel are not the same, the processor 560 of the speaker recognition server 600 can determine whether a similarity between the first speech signal and the second speech signal is equal to or greater than a first similarity.

[0139] If it is determined that the similarity between the first speech signal and the second speech signal is equal to or greater than the similarity of the first, the processor 560 of the speaker recognition server 600 can send a speaker identification notification to the display device 100 via the communication unit 510.

[0140] If it is determined that the similarity between the first speech signal and the second speech signal is less than the first similarity, the processor 560 of the speaker recognition server 600 can determine whether the similarity is equal to or greater than a second similarity that is less than the first similarity.

[0141] If the similarity is less than the second similarity, the processor 560 of the speaker recognition server 600 can send a speaker registration notification to the display device 100 via the communication unit 510 to register a speaker with a different account.

[0142] If it is determined that the similarity is equal to or greater than the second similarity, the processor 560 of the speaker recognition server 600 can send a speaker match confirmation notification to the display device 100 via the communication unit 510.

[0143] According to one embodiment, the speaker match confirmation notification can be a notification to confirm that a currently speaking speaker matches a previously stored speaker.

[0144] The processor 560 of the speaker recognition server 600 can receive a confirmation message from the display device 100 indicating that the speakers agree.

[0145] The 560 processor of the 600 speaker recognition server can update a speaker profile upon receiving the confirmation message.

[0146] The Gateway 610 can send a speech signal transmitted by the Display Device 100 to the Speaker Recognition Server 600 or the STT Server 620.

[0147] The gateway 610 can be omitted. In this case, the display device 100 can send the speech signal directly to the speaker recognition server 600 or the STT server 620.

[0148] The STT server 620 can convert the speech signal into text data.

[0149] The STT server 620 can send the converted text data to the speaker recognition server 600 or the NLP server 630.

[0150] The NLP server 630 can use the NLP engine to obtain an intent analysis result based on the text data received from the STT server 620.

[0151] The NLP server 630 can send the received result of the intent analysis to the display device 100 via the gateway 610.

[0152] The speaker database 601 can be included in the speaker recognition server 600 or provided separately from the speaker recognition server 600.

[0153] The speaker database 601 can store a speaker profile that corresponds to each speaker.

[0154] The speaker profile can contain an embedding vector that includes a plurality of speech input channels and speech features adapted to each speech input channel.

[0155] In the following, a speaker registration process can be a process in which features of the voice uttered by a particular speaker are extracted using speech recognition technology, and the extracted features are matched with speaker information and stored.

[0156] The speaker identification process can be a process in which it is determined whether a pre-registered speaker is identical to a currently speaking speaker.

[0157] The speaker registration process and the speaker identification process can be processes for providing personalized speech recognition services.

[0158] Fig. Figure 7 is a contact diagram to explain an operating procedure of a speaker recognition system according to an embodiment of the present disclosure.

[0159] With reference to Fig. 7. The control unit 170 of the display device 100 can receive a first voice command uttered by a speaker (S701).

[0160] Control 170 can display a speaker registration image on display 180 in response to a speaker registration request.

[0161] The speaker registration image can be an image used to register a speaker using a voice command uttered by the speaker for a keyword.

[0162] The 170 controller can receive an initial voice command via any one of the majority of voice input channels.

[0163] The voice input channel can be a device that receives the voice command uttered by the speaker.

[0164] The majority of voice input channels may include a microphone, headset, mobile devices or the like, contained in the remote control device 200 or the display device 100.

[0165] Fig. Figure 8A is a diagram illustrating a speaker registration image according to an embodiment of the present disclosure. Fig. 8B is a diagram explaining keywords provided via the speaker registration image. Fig. 8C is a diagram illustrating multiple speech input channels.

[0166] With reference to Fig. 8A, when receiving a speaker registration request, the display device 100 can display a speaker registration image 800 on the display 180.

[0167] The display device 100 can receive the speaker registration request via the remote control device 200.

[0168] The speaker registration screen 800 may include an utterance-triggering text 810, which prompts the speaker to make an utterance, and a keyword window 820, which contains a keyword.

[0169] A user outputs the keyword contained in keyword window 820. Keyword window 820 can contain any of a plurality of keywords, as in Fig. 8B is shown, and will be displayed sequentially according to the user's utterance.

[0170] The display device 100 can receive a voice command via a specific voice input channel, corresponding to each uttered keyword.

[0171] The display device 100 can receive a voice command corresponding to the keyword via one of the majority of in Fig. The input channels 200, 851, 853 and 855 shown in 8C are received.

[0172] The first voice input channel can be a microphone provided in the remote control device 200.

[0173] The remote control device 200 can send the voice command uttered by the speaker to the user interface 150 of the display device 100 via the wireless communication module 220.

[0174] The second speech input channel can be a microphone 851 provided in the display device 100, which directly captures the voice uttered by a speaker A.

[0175] The third voice input channel can be a microphone provided in a speaker's 853 headphones.

[0176] The headphones 853 can send a voice signal corresponding to a voice command via short-range wireless communication to the wireless communication interface 173 of the display device 100.

[0177] The fourth speech input channel can be a microphone provided in a mobile device 855, such as a speaker's smartphone. The mobile device 855 can transmit a speech command uttered by the speaker via short-range wireless communication to the wireless communication interface 173 of the display device 100.

[0178] Thus, the display device 100 can receive a voice command uttered by a speaker via various voice input channels.

[0179] Fig. 7 will be described again.

[0180] The control unit 170 of the display device 100 can send the first speech signal and first speech input channel information corresponding to the first speech command to the speaker recognition server 600 via the network interface 133 (S703).

[0181] The initial speech input channel information can include a speech input channel that specifies a source capturing the initial speech command.

[0182] The initial speech input channel information may include information that identifies the speech input channel, such as a name of a speech input channel or a model of the speech input channel.

[0183] The controller 170 can send initial speech input channel information, which includes the first speech signal corresponding to the first speech command and the speech input channel indicating that a device is receiving the first speech command, to the communication unit 510 of the speaker recognition server 600.

[0184] The processor 560 of the speaker recognition server 600 can receive an initial speech feature set that specifies features of the initial speech signal and store a speaker profile including the initial speech feature set and the initial speech input channel information in memory 530 (S705).

[0185] The Processor 560 can extract the first set of speech features, including speech features, from the initial speech signal. The speech features can include one or more frequency features, sound level, and pitch.

[0186] The 560 processor can extract speech features from the speech signal using a Mel frequency cepstrum coefficient (MFCC) scheme.

[0187] The MFCC scheme can be a scheme for segmenting a speech signal into a plurality of frequency bands and for calculating the energy in each frequency band.

[0188] The 560 processor can obtain a first embedding vector from the first speech signal using the MFCC scheme, which specifies a first set of speech features.

[0189] The 560 processor can generate a speaker profile obtained by matching the first speech feature set or the first embedding vector with the first speech input channel information.

[0190] The processor 560 can store the generated speaker profile in memory 530 or in the speaker database 601. The speaker profile can also contain speaker account information. The speaker profile can contain one or more speaker identifiers and / or speaker names. The speaker account information, along with the first speech signal and the first speech input channel information, can be sent to the speaker recognition server 600 in step S703.

[0191] The processor 560 of the speaker recognition server 600 can send the speaker registration notification to the display device 100 via the communication unit 510 (S707).

[0192] After generating the speaker profile, the processor 560 can send a speaker registration notification to the display device 100 via the communication unit 510, indicating that a speaker is registered in a personalized speech recognition service.

[0193] The control unit 170 of the display device 100 can output the speaker registration notification received from the speaker recognition server 600 (S709).

[0194] Control 170 can display the speaker registration notification on display 180.

[0195] The operating procedures S701 to S709 can be a speaker registration process in which a speech recognition service is provided only for one speaker, who is identified by recognizing the speaker's voice.

[0196] After the speaker registration process, the control unit 170 of the display device 100 can receive a second voice command uttered by the speaker (S711).

[0197] The controller 170 can receive the second voice command via any one of a plurality of voice input channels.

[0198] The control unit 170 of the display device 100 can send the second speech signal and the second speech input channel information corresponding to the second speech command to the speaker recognition server 600 via the network interface 133 (S713).

[0199] The controller 170 can send information about the second voice input channel, including the second voice signal and information about the second voice input channel indicating that a device is receiving the second voice command, to the speaker recognition server 600 via the network interface 133.

[0200] The second speech input channel information can include one or more names of a speech input channel receiving the second speech command, or a model name of the speech input channel.

[0201] In addition to the second speech signal and the second speech input channel information, the controller 170 can also send account information of a speaker registered at the display device 100 to the speaker recognition server 600.

[0202] The processor 560 of the speaker recognition server 600 can determine whether the first speech input channel and the second speech input channel are the same (S715).

[0203] The processor 560 can determine whether the first speech input channel, which is contained in the speaker profile stored in memory 530, and the second speech input channel, which is contained in the second speech input channel information, are the same.

[0204] If it is determined that the first speech input channel and the second speech input channel are the same, the processor 560 of the speaker recognition server 600 can perform a speaker identification process (S717).

[0205] The speaker identification process can be a process in which the first embedding vector stored in the speaker profile is compared with the second embedding vector of the second speech signal to determine whether the speakers match.

[0206] The 560 processor can extract the second embedding vector, which specifies the language features of the second speech signal, from the second speech signal using an MFCC scheme.

[0207] The Processor 560 can calculate a similarity between the first embedding vector and the second embedding vector, and if the calculated similarity is equal to or greater than a preset similarity, the Processor 560 can determine that a speaker of the first speech command is the same as a speaker of the second speech command.

[0208] The similarity between the first embedding vector and the second embedding vector can represent a cost factor for speaker identification. This similarity can be referred to as the speaker identification cost.

[0209] The preset similarity can be a first similarity, which is described below.

[0210] If it is determined that the first speech input channel and the second speech input channel are not the same, the processor 560 of the speaker recognition server 600 can determine whether a similarity between the first speech signal and the second speech signal is equal to or greater than a first similarity (S719).

[0211] The 560 processor can calculate a similarity between the first embedding vector of the first speech signal and the second embedding vector of the second speech signal.

[0212] If it is determined that the similarity between the first speech signal and the second speech signal is equal to or greater than the similarity of the first, the processor 560 of the speaker recognition server 600 can send a speaker identification notification via the communication unit 510 to the display device 100 (S721).

[0213] The speaker identification notification can be a notification indicating that a speaker of the first voice command and a speaker of the second voice command are the same, in order to identify a speaker.

[0214] Then the display device 100 can receive an analysis result of the second voice command from the NLP server 630 and output the received analysis result.

[0215] Fig. Figure 9 is a diagram illustrating a case in which a speaker who is speaking is identified via the same speech input channel according to an embodiment of the present disclosure.

[0216] If it is determined that the similarity between the first speech signal and the second speech signal is equal to or greater than the similarity of the first, the display device 100 can display an analysis result 900 of the second speech command on the display 180.

[0217] Analysis result 900 can include a search result of content as an intent analysis result of the second voice command.

[0218] The speaker recognition server 600 can send a notification indicating that the speaker has been identified to the NLP server 630. Based on the text data of the second voice command, the NLP server 630 can receive an analysis result of the second voice command according to the received notification. The NLP server 630 can send the analysis result to the display device 100.

[0219] Fig. Section 7 will be described again.

[0220] If it is determined that the similarity between the first speech signal and the second speech signal is less than the first similarity (S719), the processor 560 of the speaker recognition server 600 can determine whether the similarity is equal to or greater than a second similarity that is less than the first similarity (S723).

[0221] The second similarity could be a standard that covers a case in which a speech signal uttered by the same speaker is modified depending on the characteristics of microphones from different speech input channels.

[0222] If the similarity is less than the second similarity, the processor 560 of the speaker recognition server 600 can send a speaker registration notification to register a speaker with a different account (new account) via the communication unit 510 to the display device 100 (S725).

[0223] This means that if the similarity is less than the second similarity, the processor 560 can determine that a speaker of a first speech command and a speaker of a second speech command are different from each other.

[0224] The processor 560 can send a speaker registration notification to the display device 100 to initiate speaker registration with another account.

[0225] Fig. Figure 10 is a diagram illustrating an example in which a speaker registration notification is issued to initiate speaker registration with another account according to an embodiment of the present disclosure.

[0226] With reference to Fig. 10. If the similarity is less than the second similarity, the processor 560 of the speaker recognition server 600 can send the speaker registration notification for speaker registration with another account via the communication unit 510 to the display device 100.

[0227] The display device 100 can display a speaker registration notification 1000 on the display 180.

[0228] Upon receiving a confirmation command via speaker registration notification 1000, the display device 100 can display the speaker registration image 800, as shown in Fig. 8A shown.

[0229] Fig. Section 7 will be described again.

[0230] If it is determined that the similarity is equal to or greater than the second similarity (S723), the processor 560 of the speaker recognition server 600 can send a speaker match confirmation notification to the display device 100 via the communication unit 510 (S727).

[0231] According to one embodiment, the speaker match confirmation notification can be a notification to confirm that a currently speaking speaker matches a previously stored (or registered) speaker.

[0232] The processor 560 of the speaker recognition server 600 can receive an acknowledgment message from the display device 100 indicating that the speakers agree (S729).

[0233] The 560 processor of the 600 speaker recognition server can update the speaker profile upon receiving the confirmation message (S731).

[0234] Fig. Figures 11A to 11C are diagrams illustrating a process for updating an existing speaker profile by adding information to a new speech input channel for the same speaker according to an embodiment of the present disclosure.

[0235] With reference to Fig. 11A The display device 100 can display an initial pop-up window 1110 on the display 180 when it receives a speaker matching confirmation notification from the speaker recognition server 600.

[0236] In the Fig. 11A to 11C assumes that a pre-registered speech input channel in a speaker profile is the remote control device 200 and a new speech input channel is a headset 853.

[0237] The first pop-up window 1110 may contain text asking whether a pre-registered speaker and a current speaker are similar and match.

[0238] The display device 100 can display a second pop-up window 1130 for easy registration of a new voice input channel on the display 180 when a command to select an approval button is received via the first pop-up window 1110.

[0239] The speaker recognition server 600 can respond to a command to select an consent button via the first pop-up window 1110 and send a notification to the display device 100 to display the second pop-up window 1130 for easy registration of a new speech input channel.

[0240] The second pop-up window 1130 can be displayed to assign a channel environment to the speaker for the voice received via a new voice input channel, even if the speaker has been identified.

[0241] If the consent button is selected via the second pop-up window 1130, the display device 100 can display a keyword initiation window 1150, which induces the utterance of a keyword via a new speech input channel on the display 180, as shown in Fig. 11B shown.

[0242] The keyword induction window 1150 can contain two keywords.

[0243] The speaker can utter the two keywords contained in keyword induction window 1150 one after the other via headset 853.

[0244] The headset 853 can send voice commands corresponding to the uttered keywords to the display device 100, and the display device 100 can send the voice commands and the voice input channel information to the speaker recognition server 600.

[0245] The Speaker Recognition Server 600 can extract speech features from voice commands and generate an embedding vector based on the extracted speech features.

[0246] The speaker recognition server 600 can add the generated embedding vector and the information about the headset 853 as a new speech input channel to a pre-stored speaker profile.

[0247] This allows the speaker profile to be updated.

[0248] According to one embodiment of the present disclosure, the speaker can be recognized normally even when the speaker expresses his voice via the remote control device 200 or the headset 853 as a new speech input channel.

[0249] This means that, according to one embodiment, the performance of speaker recognition can be improved even if the speaker expresses their voice via different speech input channels.

[0250] A first embedding vector, corresponding to the first speech input channel stored in the speaker profile, and a second embedding vector, corresponding to a new second speech input channel, can be different.

[0251] A difference between the first embedding vector and the second embedding vector can be stored in the speaker profile.

[0252] According to another embodiment of the present disclosure, upon receiving a command to select an approval button via the first pop-up window 1110, the display device 100 can display a third pop-up window 1170 in which the utterance of a keyword is requested via the remote control device 200, which is pre-registered.

[0253] The speaker recognition server 600 can respond to the receipt of a command to select the consent button via the first pop-up window 1110 and send a notification that causes an additional utterance of the keyword to display the window 1170 via the pre-registered remote control device 200.

[0254] The display device 100 can send voice commands for keywords uttered by the speaker to the speaker recognition server 600 via the remote control device 200.

[0255] If the speaker identification process detects that the speaker is a previously registered speaker, the speaker recognition server 600 can store information about speech characteristics of the voice commands uttered via the headset 853 and the headset 853 as a voice input channel in the speaker profile.

[0256] According to one embodiment of the present disclosure, the method described above can be implemented as processor-readable code on a medium on which a program is recorded. Examples of processor-readable media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and the like.

[0257] The display device described above is not limited to the configuration and method of the embodiments described above, but the embodiments described above can be set up by selectively combining all or part of each embodiment, so that various modifications can be made.

Claims

[1] Speaker recognition device comprising: a communication interface configured to communicate with an electronic device; a memory configured to store a speaker profile comprising a first speech feature set including speech features of a first speech signal and a first speech input channel matching the first speech signal; and a processor configured to receive from an electronic device a second speech signal corresponding to a reception of a voice command uttered by a speaker and information via a second speech input channel indicating a device receiving the voice command, Determining a similarity between the first speech feature set and a second speech feature set including speech features of the second speech signal based on the second speech input channel not being the first speech input channel stored in the memory, and Sending a notification indicating that the speaker has been identified to the electronic device based on the obtained similarity being equal to or greater than a first similarity. [2] The speaker recognition apparatus according to claim 1, wherein the processor compares a first embedding vector indicative of the first speech feature set with a second embedding vector indicative of the second speech feature set to obtain the similarity. [3] The speaker recognition device according to claim 1 or 2, wherein based on the similarity being smaller than the first similarity and equal to or greater than a second similarity which is smaller than the first similarity, the processor is configured to send a notification for checking that the speaker matches a pre-registered speaker to the electronic device. [4] The speaker recognition device according to any one of claims 1 to 3, wherein based on the similarity being smaller than the first similarity and being equal to or greater than a second similarity which is smaller than the first similarity, the processor is configured to send a notification for registering the second speech input channel to the electronic device. [5] The speaker recognition device according to claim 4, wherein the processor is configured to obtain additional speech features of an additional speech signal corresponding to an additional speech command received via the second speech input channel from the electronic device and to update the information about the second speech input channel and the additional speech features in the speaker profile. [6] Speaker recognition device according to one of claims 1 to 5, wherein based on the similarity being less than the first similarity and being equal to or greater than a second similarity which is less than the first similarity, the processor is configured to send a notification causing an additional utterance of a keyword to the electronic device via the first speech input channel. [7] The speaker recognition device according to any one of claims 3 to 6, wherein based on the similarity being smaller than the second similarity, the processor is configured to send a notification for registering a new speaker to the electronic device. [8] The speaker recognition device according to any one of claims 1 to 7, wherein each of the first speech input channel and the second speech input channel comprises one of a remote control device, the electronic device and a mobile device, each including a microphone. [9] A method of operating a speaker recognition device, the method comprising: Storing a speaker profile comprising a first speech feature set including speech features of a first speech signal and a first speech input channel matching the first speech signal; Receiving, from an electronic device, a second voice signal corresponding to a receipt of a voice command uttered by a speaker and information over a second voice input channel indicating a device receiving the voice command; Determining a similarity between the first speech feature set and a second speech feature set including speech features of the second speech signal based on the second speech input channel not being the first speech input channel stored in the memory; and Sending a notification indicating that the speaker has been identified to the electronic device based on the obtained similarity being equal to or greater than a first similarity. [10] The method of claim 9, wherein obtaining the similarity comprises comparing a first embedding vector indicative of the first speech feature set with a second embedding vector indicative of the second speech feature set to obtain the similarity. [11] The method of claim 9 or 10, further comprising, based on the similarity being less than the first similarity and being equal to or greater than a second similarity that is less than the first similarity, sending a notification to the electronic device to verify that the speaker matches a pre-registered speaker. [12] The method of any one of claims 9 to 11, further comprising, based on the similarity being less than the first similarity and being equal to or greater than a second similarity that is less than the first similarity, sending a notification to register the second voice input channel to the electronic device. [13] The method of claim 12, further comprising: Obtaining additional speech features of an additional speech signal corresponding to an additional speech command received via the second speech input channel from the electronic device and Updating the information about the second speech input channel and the additional speech features in the speaker profile. [14] The method of any one of claims 9 to 13, further comprising, based on the similarity being less than the first similarity and being equal to or greater than a second similarity that is less than the first similarity, sending a notification causing an additional utterance of a keyword to the electronic device via the first voice input channel. [15] The method of any one of claims 11 to 14, further comprising, based on the similarity being less than the second similarity, sending a notification to register a new speaker to the electronic device.

Images