Voice recognition device and method of operating the same

Abstract

The present disclosure relates to a speech recognition device and an operating method thereof for analyzing user input based on content and generating and outputting a response, the operating method comprising: receiving an audio signal and performing speech recognition on the audio signal; acquiring content information of the content being executed; based on the content information, analyzing the user input from a speech recognized by performing speech recognition; generating a response based on the analyzed user input and content information; and outputting the response.

Description

Technical Field

[0001] The present disclosure relates to a voice recognition device and an operating method thereof, and, for example, to a voice recognition method and device for analyzing a user input based on content being executed and generating and outputting a response.

[0002] The present disclosure also relates to an artificial intelligence (AI) system and its applications for simulating functions such as recognition and determination using a machine learning algorithm such as deep learning. Background Art

[0003] Artificial intelligence (AI) systems are computer systems with human-level intelligence. Unlike existing rule-based intelligent systems, AI systems are able to learn autonomously, make decisions, and become intelligent. As the use of AI systems has more accurately improved the recognition rate and understanding of user tastes, existing rule-based intelligent systems have gradually been replaced by deep learning-based AI systems.

[0004] AI technologies include machine learning (deep learning) and meta-technologies that use machine learning.

[0005] Machine learning may refer to an algorithmic technology that autonomously classifies / autonomously learns features of multiple input data. Meta-technology may refer to a technology that uses machine learning algorithms such as deep learning to simulate functions such as recognition and determination of the human brain, and includes language understanding, visual understanding, inference / prediction, knowledge expression, motion control, etc.

[0006] The various fields where AI technology is applied are as follows. Language understanding is a technology that recognizes and applies / processes human language / characters, and includes natural language processing, machine translation, dialogue systems, inquiry and response, speech recognition / synthesis, etc. Visual understanding is a technology that recognizes and processes objects like human vision, and includes object recognition, object tracking, image search, human recognition, scene understanding, spatial understanding, image enhancement, etc. Inference / prediction is a technology that determines information to perform logical inference and prediction, and includes knowledge / probability-based inference, optimization prediction, preference-based planning, recommendation, etc. Knowledge expression is a technology that automatically processes human experience information into knowledge data, and includes knowledge construction (data creation / classification), knowledge management (data utilization), etc. Motion control is a technology that controls the autonomous driving of vehicles and the movement of robots, and includes motion control (navigation, collision and driving), operation control (behavior control), etc.

[0007] Recently, with the development of electronic devices for performing various functions, such as smart phones, electronic devices with voice recognition functions for improving operability have been introduced. The voice recognition function allows users of electronic devices to easily control the device by voice without operating a separate button or touching a touch module.

[0008] For the voice recognition function, various AI technologies can be used. Through language understanding, the content of the user's speech can be accurately recognized, and the user's intention included in the speech content can be detected. In addition, through visual understanding, the content used by the user can be understood, and through inference / prediction, a suitable response according to the user's intention and the content being executed by the user can be generated. In addition, through motion control, the generated response can be output in a suitable form. Summary of the invention

[0009] Technology Solutions

[0010] According to one aspect of the present disclosure, a method for operating a voice recognition device includes: receiving an audio signal and performing voice recognition; acquiring content information of content being executed; based on the content information, analyzing user input from a voice recognized by performing voice recognition; generating a response based on the analyzed user input and the content information; and outputting the response.

[0011] Beneficial Effects

[0012] A speech recognition device and an operating method thereof are provided for analyzing user input based on content and generating and outputting a response. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent from the following description in conjunction with the accompanying drawings, in which:

[0014] Figure 1 is a diagram illustrating a speech recognition method according to an embodiment;

[0015] Figure 2 is a block diagram illustrating an example speech recognition method according to an embodiment;

[0016] Figure 3 is a flowchart illustrating an operation method of a speech recognition device according to an embodiment;

[0017] Figure 4 , Figure 5 and Figure 6 is a diagram illustrating a speech recognition system according to an embodiment;

[0018] Figure 7 is a block diagram illustrating a method for acquiring content information by a voice recognition device according to an embodiment;

[0019] Figure 8 is a block diagram illustrating a method for recognizing a user input by a speech recognition device according to an embodiment;

[0020] Fig. 9 is a block diagram illustrating a method for acquiring context information by a speech recognition device according to an embodiment;

[0021] Fig.10 is a block diagram illustrating a method for generating a response by a speech recognition device according to an embodiment;

[0022] Fig.11 is a flow chart illustrating a method for determining the relevance of a response by a speech recognition device according to an embodiment;

[0023] Fig.12 is a block diagram illustrating a method for determining an output form of a response by a speech recognition device according to an embodiment;

[0024] Fig.13 is a block diagram showing a speech recognition device according to an embodiment;

[0025] Fig.14 is a block diagram showing a speech recognition device according to an embodiment;

[0026] Fig.15 is a diagram showing an example of outputting a response that varies depending on the content being executed;

[0027] Fig.16 is a diagram showing an example in which a speech recognition device outputs a response that varies according to context information;

[0028] Fig.17 is a diagram showing another example of outputting a response that varies depending on the content being executed;

[0029] Fig.18 is a diagram showing an example of determining the relevance of a response;

[0030] Fig.19 is a diagram showing an example form of a response output according to an embodiment; and

[0031] Fig. 20 is a diagram illustrating another example in which a voice recognition apparatus according to an embodiment outputs a response that varies according to context information.

[0032] Best Mode

[0033] A speech recognition device and an operating method thereof are provided for analyzing user input based on content and generating and outputting a response.

[0034] Additional aspects will be set forth in part in the description which follows and, in part, will be apparent from the description.

[0035] According to one aspect of the present disclosure, a method for operating a voice recognition device includes: receiving an audio signal and performing voice recognition; acquiring content information of content being executed; analyzing user input from a voice recognized by performing voice recognition based on the content information; generating a response based on the analyzed user input and the content information; and outputting the response.

[0036] According to another aspect of the present disclosure, a speech recognition device includes: a receiver, including a receiving circuit configured to receive an audio signal; at least one processor, configured to perform speech recognition on the audio signal, obtain content information of the content being executed, analyze user input from a speech recognized by performing speech recognition based on the content information, and generate a response based on the analyzed user input and the content information; and an output unit, including an output circuit configured to output the response.

[0037] According to another aspect of the present disclosure, a non-transitory computer-readable recording medium having recorded thereon a program including instructions, wherein the instructions, when executed by a processor, cause a device to perform operations including the following steps: receiving an audio signal and performing speech recognition; obtaining content information of content being executed; analyzing user input from speech recognized by performing speech recognition based on the content information; generating a response based on the analyzed user input and the content information; and outputting the response. DETAILED DESCRIPTION

[0038] Reference will now be made in more detail to example embodiments, wherein examples of example embodiments are shown in the accompanying drawings. In the accompanying drawings, parts irrelevant to the description may be omitted to clearly describe the example embodiments, and throughout the disclosure, the same reference numerals represent the same elements. In this regard, the present example embodiments may have different forms and should not be construed as being limited to the description set forth herein. Therefore, the following will only describe example embodiments with reference to the accompanying drawings to explain various aspects. As used herein, the term "and / or" includes any and all combinations of one or more associated listed items. When an expression such as "at least one of ... " is after a column of elements, it is to modify the entire column of elements rather than a single element modifying the column.

[0039] Function blocks and various processing steps may be utilized to represent some embodiments of the present disclosure. Some or all of these function blocks may be implemented by various numbers of hardware and / or software configurations for performing specific functions. For example, the function blocks of the present disclosure may be implemented by one or more microprocessors or by circuit configurations for specific functions. In addition, for example, the function blocks of the present disclosure may be implemented by various programming or scripting languages. The function blocks may be implemented by algorithms executed in one or more processors. In addition, the present disclosure may employ related technologies for electronic environment installation, signal processing, and / or data processing.

[0040] In addition, the connecting lines or connecting members between the components shown in the drawings show functional connections and / or physical or circuit connections. In actual equipment, the connections between the components can be represented by various functional connections, physical connections or circuit connections that can be replaced or added.

[0041] In addition, terms such as "unit", "module", etc. may refer to a unit that performs at least one function or operation, and the unit may be implemented as hardware or software or as a combination of hardware and hardware. The "unit" or "module" may be implemented by a program stored in an addressable storage medium and executable by a processor.

[0042] For example, a “unit” or “module” may be implemented by a component (such as a software component, an object-oriented software component, a class component, and a task component), a process, a function, a property, a procedure, a subroutine, a program code fragment, a driver, firmware, microcode, a circuit, data, a database, a data structure, a table, an array, and / or a variable.

[0043] Figure 1 is a diagram illustrating a speech recognition method according to an embodiment.

[0044] Reference Figure 1 According to the embodiment, the speech recognition device 100 may receive an audio signal including a speech signal emitted by the user 10, and perform speech recognition on the speech signal included in the received audio signal. Thereafter, the speech recognition device 100 may understand the user input included in the speech signal, and generate and output a response in response to the user input. Through this process, the user 10 may control the speech recognition device 100 using the speech signal.

[0045] In addition, in addition to the consistent analysis and understanding of user input through the combination of speech recognition technology and artificial intelligence (AI) technology, the speech recognition device 100 according to the embodiment can also generate and output a response suitable for the intention of the user 10 and the situation in which the speech recognition device 100 operates. For example, the speech recognition device 100 according to the embodiment can be an AI system based on deep learning. According to the embodiment, the speech recognition device 100 can use AI technology to recognize, apply and process human language and infer and predict the situation in which the speech recognition device 100 operates.

[0046] In an embodiment, the speech recognition device 100 may receive an audio signal in an idle state, or may receive an audio signal in a specific case. For example, the speech recognition device 100 may receive an audio signal when the content is being executed. In this case, the speech recognition device 100 may analyze the user input based on the content being executed in the speech recognition device 100, rather than only analyzing the user input from the voice recognized by performing speech recognition on the audio signal, to more accurately understand the intention according to the speech of the user 10. In addition, even when a response to the user input is generated and output, a suitable response may be output in a suitable form by generating a response based on the content being executed and the situation of the speech recognition device 100 operation.

[0047] Reference Figure 1 , the video content 20 in which a panda appears is being executed in the speech recognition device 100. In this case, the speech recognition device 100 can relatively accurately understand the intention of the user 10 by analyzing the user input based on the video content 20 being executed. For example, when the user 10 wants to know where the panda that appears in the video content 20 lives, the user 10 can ask "where does that panda live?". In this case, in the case of an existing speech recognition device that receives an audio signal and performs speech recognition on the audio signal, the existing speech recognition device cannot detect which panda is "that panda" indicated by the user, and therefore, the existing speech recognition device cannot accurately understand the user's intention. On the other hand, the speech recognition device 100 according to the embodiment can understand the user 10 asking "where does the panda that appears in the video content 20 being executed live?" by analyzing the user input based on the video content 20 being executed.

[0048] In addition, the speech recognition device 100 can generate a suitable response based on the video content 20 being executed in response to the user input. For example, the speech recognition device 100 can infer based on the video content 20 being executed that the "panda appearing in the video content 20 being executed" lives in the "Y Zoo".

[0049] Figure 1A speech recognition device 100 is shown, wherein the speech recognition device 100 is used to receive an audio signal, understand a user input included in the received audio signal, and generate and output a response in response to the user input, however, the speech recognition device 100 is not limited thereto, and may also receive various input signals such as gestures, characters, and / or videos, understand the user input included in the corresponding signal, and generate and output a response in response to the user input. For example, an electronic device including the speech recognition device 100 may receive signals such as gestures, characters, and / or videos from a user through various sensors and / or user interfaces, understand the user input included in the received signal, and generate and output a response in response to the user input.

[0050] In addition, the voice recognition device 100 may output a response in a suitable form in consideration of the situation in which the voice recognition device 100 operates, the video content being executed, etc. For example, the voice recognition device 100 may output a response based on the situation in which the voice recognition device 100 operates without interrupting the video content being viewed by the user 10. The method of outputting a response will be described in more detail below.

[0051] Figure 2 is a block diagram illustrating a detailed speech recognition method according to an embodiment.

[0052] Figure 2 The speech recognition method shown in includes reference Figure 1 In addition, in the following, as an example, the speech recognition device 100 is described as being executed as follows Figure 1 The same situation occurs with Panda video content 20.

[0053] Reference Figure 2 , the voice recognition apparatus 100 according to the embodiment receives an audio signal including the voice of the user 10 while executing the video content 20 in which the panda appears. For example, when the user 10 asks "Where does that panda live?", the voice recognition apparatus 100 receives an audio signal including the corresponding voice.

[0054] According to an embodiment, the speech recognition device 100 may perform speech recognition 210 on the received audio signal. The speech recognition device 100 may receive user input by performing speech recognition 210 on the received audio signal. For example, when the user 10 asks "Where does that panda live?", the speech recognition device 100 may understand the meaning of the sentence "Where does that panda live?" as its own meaning, and receive the meaning of the sentence as user input. The speech recognition 210 may be, for example, a process of analyzing and understanding the speech signal of the user 10 in a consistent manner.

[0055] In this case, the speech recognition device 100 can use AI technology to identify, apply and process the language of the speaker included in the audio signal. In an embodiment, the speech recognition device 100 can improve the accuracy of speech recognition 210 by learning (e.g., deep learning) the personal characteristics of the user 10 (e.g., intonation, language habits, frequently used words and expressions, etc.) or the language understanding of the speech signal (e.g., natural language processing, etc.).

[0056] According to an embodiment, the voice recognition device 100 may obtain content information of the content being executed (e.g., video content 20 in which a panda appears) via content information acquisition 220. The content information may include story information and execution information of the content being executed. The story information of the content may include information about the content itself, such as an overview of the content, the state of objects included in the content, the relationship between objects, etc. In addition, the execution information of the content may include information indicating the execution state of the content, such as the name of the content and the progress of the content. For example, the voice recognition device 100 may obtain information related to the overview, animals and plants in the content, the relationship between humans and animals and plants, the name of the video content 20, the total playback time, the current progress, etc. for the content being executed.

[0057] In this case, the voice recognition apparatus 100 may use AI technology to acquire wide and deep content information via the content information acquisition 220. For example, the voice recognition apparatus 100 may improve the degree of understanding of the scene of the content being executed through visual understanding of the content.

[0058] In addition, the content information acquisition 220 and the speech recognition 210 may be performed on the audio signal in parallel or sequentially. When the content information acquisition 220 and the speech recognition 210 are performed sequentially, the speech recognition 210 or the content information acquisition 220 may be performed first. In addition, when the content is being executed, the content information acquisition 220 may be performed continuously regardless of other processing. According to an embodiment, the speech recognition apparatus 100 may acquire content information by analyzing the content being executed, or acquire information related to the content being executed from the outside (e.g., a server, a social network service (SNS), a database, etc.).

[0059] According to an embodiment, the speech recognition device 100 may analyze the user input via the user input analysis 230 based on the speech recognition result and the acquired content information. Here, the user input analysis 230 is set so as to more accurately understand the user 10's speaking intention than the consistent analysis and understanding of the user 10's voice signal in the speech recognition 210. For example, the meaning of the sentence "Where does that panda live?" is understood as its own meaning through the speech recognition 210, but can be understood as the user 10 asking "Where does the panda appearing in the video content 20 being executed live?" through the user input analysis 230 based on the acquired content information.

[0060] Compared with the case where AI technology is not used, the voice recognition device 100 can use AI technology to more accurately understand the user input. For example, the voice recognition device 100 can relatively accurately understand the user input by logically inferring and predicting the intention of the user 10 based on the content information.

[0061] According to an embodiment, the speech recognition device 100 may generate a response via the response generation 240 based on the acquired content information in response to the recognized user input. The speech recognition device 100 may generate a suitable response desired by the user 10 based on the acquired content information. In this case, the speech recognition device 100 may consider not only the content information but also the context information of the speech recognition device 100. The context information of the speech recognition device 100 may include information related to the position, movement, peripheral environment of the speech recognition device 100, whether the speech recognition device 100 is connected to another electronic device, the characteristics of the user using the speech recognition device 100, etc.

[0062] In addition, the speech recognition device 100 can use various methods to generate the response 240. For example, the speech recognition device 100 can determine the type of content through the content information, generate the response through a movie analysis scheme (such as plot analysis, character analysis, and scene analysis) when the content is a movie, and generate the response through a music analysis scheme (such as a music information retrieval system, chord analysis, and pitch analysis) when the content is music. Since the response generated by this process is generated by understanding the user input of the user's 10 true intention, the response is very likely (e.g., more likely) to be the response expected by the user 10. For example, the speech recognition device 100 can obtain information indicating that the content being executed is a documentary related to the life of pandas through the content information, detect from the database that the scene being watched by the user 10 was filmed at the Y zoo, and generate the response "Pandas live in the Y zoo".

[0063] In this case, the speech recognition device 100 can use AI technology to generate the best response. For example, the speech recognition device 100 can use a database through knowledge management to generate a response closest to the response expected by the user 10 or determine the relevance of the response. Here, the database can be a knowledge-based database.

[0064] According to an embodiment, the speech recognition device 100 may output the generated response. In this case, the speech recognition device 100 may output the response in a suitable form in consideration of the context information of the speech recognition device 100, etc. For example, when the explanation of the life of a panda is being output in the content being executed at the time point of outputting the response or the user 10 is in an environment where the response cannot be correctly heard due to the noisy area around the speech recognition device 100, the speech recognition device 100 may output the response as subtitles instead of voice. In this case, the speech recognition device 100 may use AI technology to determine a relatively suitable output form of the response.

[0065] Hereinafter, the speech recognition method and speech recognition device according to various exemplary embodiments of the present disclosure will be described with reference to the accompanying drawings. As described above, the speech recognition method and speech recognition device according to the embodiments of the present disclosure can perform speech recognition based on the content being executed, and can provide a suitable response in a suitable form according to the speech recognition.

[0066] Figure 3 is a flowchart illustrating an operation method of a voice recognition apparatus according to an embodiment.

[0067] Reference Figure 3 , in operation 310, the voice recognition apparatus 100 receives an audio signal and performs voice recognition on the received audio signal.

[0068] In operation 320, the voice recognition apparatus 100 acquires content information of the content being executed. The content information may include story information and execution information of the content being executed.

[0069] although Figure 3 It is illustrated that operation 310 is performed and then operation 320 is performed, but operations 310 and 320 may be performed sequentially or in parallel, and when operations 310 and 320 are performed sequentially, operation 320 may be performed and then operation 310 may be performed.

[0070] In addition, despite the Figure 3 Although not shown in the figure, the voice recognition apparatus 100 may further perform an operation of acquiring context information related to the situation in which the voice recognition apparatus 100 operates.

[0071] In operation 330, the voice recognition apparatus 100 analyzes user input from the voice recognized by performing voice recognition based on the content information. According to an embodiment, the voice recognition apparatus 100 may perform natural language understanding for the recognized voice based on the content information. In more detail, natural language understanding for the recognized voice may be performed, and then the natural language understanding for the recognized voice may be corrected based on the content information. In addition, the voice recognition apparatus 100 may analyze user input based on context information.

[0072] In operation 340, the speech recognition device 100 generates a response based on the analyzed user input and content information. According to an embodiment, the speech recognition device 100 can determine the relevance of the response, and correct the response when it is determined that the response is not relevant. In this case, the general relevance, user relevance, etc. of the response can be determined. Optionally, the speech recognition device 100 can generate a response based on context information.

[0073] In operation 350, the voice recognition apparatus 100 outputs a response. According to an embodiment, the voice recognition apparatus 100 may determine an output form of a response based on content information and / or context information.

[0074] According to an embodiment, the voice recognition apparatus 100 may perform voice recognition based on the content being executed, and provide a suitable response in a suitable form according to the voice recognition.

[0075] Figure 4 , Figure 5 and Figure 6 is a diagram illustrating a speech recognition system according to an embodiment.

[0076] like Figure 4 As shown in, the speech recognition system according to the embodiment may include a speech recognition device 100. For example, the speech recognition device 100 may be a mobile computing device or a non-mobile computing device, such as, but not limited to, a smart phone, a tablet personal computer (PC), a PC, a smart TV, a personal digital assistant (PDA), a laptop computer, a media player, a micro server, a global positioning system (GPS) device, a wearable device, an e-book terminal, a digital broadcast terminal, a navigation machine, an information kiosk, an MP3 player, a digital camera, an electronic control device, a central information display (CID) in a vehicle, etc., but not limited thereto. The speech recognition system may not be an independent device, but is included in another device as a component. For example, the speech recognition system may be included in a smart phone or in a CID included in an electronic control device or a vehicle.

[0077] The voice recognition apparatus 100 according to an embodiment may receive an audio signal including a voice signal uttered by the user 10, and perform voice recognition on the voice signal included in the received audio signal. Thereafter, the voice recognition apparatus 100 may understand the user input from the recognized voice, and generate and output a response in response to the user input.

[0078] Alternatively, if Figure 5 As shown in , the speech recognition system according to the embodiment may include a wearable device 100-1 and an electronic device 110 connected to (eg, coupled to) the wearable device 100-1. The wearable device 100-1 and the electronic device 110 may be connected in a wired or wireless manner.

[0079] For example, the wearable device 100-1 may be a mobile computing device or a non-mobile computing device having a shape such as, but not limited to, a watch, a watchband, glasses, a headset, earphones, etc., but not limited thereto. In addition, the electronic device 110 connected to the wearable device 100-1 may be a mobile computing device or a non-mobile computing device, such as, for example, but not limited to, a smart phone, a tablet PC, a PC, a smart TV, a PDA, a laptop computer, a media player, a micro server, a GPS device, a wearable device, an e-book terminal, a digital broadcast terminal, a navigation machine, an information kiosk, an MP3 player, a digital camera, a CID or an electronic control device in a vehicle, etc., but not limited thereto.

[0080] The wearable device 100-1 according to the embodiment may receive an audio signal including a voice signal emitted by the user 10, and transmit the received audio signal to the electronic device 110. Alternatively, the wearable device 100-1 may receive an audio signal including a voice signal emitted by the user 10 and transmit a voice signal detected from the received audio signal to the electronic device 110. Alternatively, the wearable device 100-1 may receive an audio signal including a voice signal emitted by the user 10 and transmit a feature of the voice signal detected from the received audio signal to the electronic device 110. It will be understood that the aforementioned options are merely examples, and the present disclosure is not limited thereto.

[0081] The electronic device 110 may perform voice recognition based on a signal received from the wearable device 100-1. For example, the electronic device 110 may perform voice recognition on a voice signal detected from an audio signal input from the wearable device 100-1. The electronic device 110 may output a response based on the voice recognition, or send the voice recognition result to the wearable device 100-1 so that the wearable device 100-1 outputs a response based on the voice recognition.

[0082] Alternatively, if Figure 6As shown in , the speech recognition system according to the embodiment may include a speech recognition device 100-2 and a server 120 connected to the speech recognition device 100-2 (eg, operable to communicate with the speech recognition device 100-2). The speech recognition device 100-2 and the server 120 may be connected to each other in a wired or wireless manner.

[0083] The voice recognition device 100-2 may transmit an audio signal including a voice signal emitted by the user 10, a voice signal detected from the audio signal, or a feature of the voice signal to the server 120, and the server 120 may receive the audio signal, the voice signal, or the feature and perform voice recognition. In addition, the server 120 may transmit a voice recognition result to the voice recognition device 100-2 so that the voice recognition device 100-2 outputs a response according to the voice recognition.

[0084] The speech recognition system according to the embodiment may be an AI system based on deep learning. Using AI technology, the speech recognition system according to the embodiment may recognize and process human language by inferring and predicting the operation of the speech recognition device.

[0085] In the following, reference will be made to Figures 7 to 12 Various exemplary operation methods of the speech recognition device according to the embodiment are described in more detail. Figure 1 The same case as in the case of the video content 20 in which a panda appears is being executed is described as an example.

[0086] Figure 7 is a block diagram illustrating a method for acquiring content information by the voice recognition apparatus 100 according to an embodiment.

[0087] As described above, the content information may include content story information 710 related to the story of the content being executed and / or content execution information 720 related to the execution of the content being executed. Figure 7 According to the embodiment, the speech recognition device 100 may acquire content information using analysis 701 of the content being executed and databases 702 and 703. For example, the databases 702 and 703 may include a content-related database 702 including data about the story of the content, the format of the content, etc. and a content execution status-related database 703 including data indicating the status of the content being executed.

[0088] The content-related database 702 may be, for example, a knowledge-based database, and may be stored in the speech recognition apparatus 100, or may be stored in an external device so that the speech recognition apparatus 100 uses the content-related database 702 by accessing the external device via a communication interface (not shown). The content execution state-related database 703 is a database generated, stored, and managed according to the execution of the content, and therefore, the content execution state-related database 703 may be stored in a device for executing the content or may be stored in another device connected to the device for executing the content. Therefore, when the speech recognition apparatus 100 executes the content, the content execution state-related database 703 may be stored in the speech recognition apparatus 100 or may be stored in a storage device connected to the speech recognition apparatus 100.

[0089] The speech recognition apparatus 100 according to the embodiment may analyze 701 the content being executed and acquire information related to the content being executed from the content-related database 702, thereby acquiring content story information 710. Figure 7 , the speech recognition device 100 can use AI technology to perform visual understanding on the content being executed, use a scene analysis scheme to analyze the content being executed, and use metadata of the content, etc. to understand the story of the content. For example, the speech recognition device 100 can detect that a panda appears in the content through visual understanding, and detect that the content is related to the life of a panda in a zoo through scene analysis. In addition, the speech recognition device 100 can obtain information indicating that the name of the program is "Going to the Zoo", the number of the broadcast episode is "Episode 12", and the story of the episode is "A Study of the Life of Pandas in the Zoo".

[0090] As described above, the acquired content story information 710 may include information about the content itself, such as an overview of the content, the states of objects included in the content, and the relationships between the objects. Figure 7 , according to the content being executed in the voice recognition device 100, a panda appears in an indoor enclosure and people carrying cameras appear outside the enclosure, and the people are spectators watching the pandas in the zoo.

[0091] The speech recognition apparatus 100 according to the embodiment can obtain the content execution information 720 from the content-related database 702 and the content execution status-related database 703. Figure 7, the speech recognition device 100 may obtain information indicating the following information from the content-related database 702: the name of the program being executed is "Going to the Zoo", the number of the broadcast episode is "Episode 12", and the story of the episode is "A Study of the Life of Pandas in the Zoo". In addition, the speech recognition device 100 may obtain information indicating the following information from the content execution status-related database 703: the broadcast time of the content being executed is one hour, twelve minutes of the broadcast time has passed, the sound output is 80 / 100, subtitles exist but are not currently used, and the content being executed is video on demand (VOD).

[0092] As described above, the acquired content execution information 720 may include summary progress, main story, etc. Figure 7 , according to the content being executed in the voice recognition device 100, the summary progress is 100 / 500 in the whole story, and the main story is related to the life of the panda in the zoo.

[0093] Figure 8 is a block diagram illustrating a method used by the voice recognition apparatus 100 to recognize a user input according to an embodiment.

[0094] The speech recognition apparatus 100 according to the embodiment receives an audio signal including a speech signal uttered by the user 10, and performs speech signal analysis 810 on the speech signal included in the received audio signal. Figure 8 , when the user 10 asks “Where does the panda live?”, the speech recognition apparatus 100 may extract only the speech signal “Where does the panda live?” of the user 10 from the received audio signal. In this process, the speech recognition apparatus 100 may detect through the speech signal analysis 810 that the user 10 is a male, the age of the user 10 is about thirty years old, and the emotional state of the user 10 is calm.

[0095] The speech recognition apparatus 100 according to the embodiment may perform sentence analysis 820 to perform natural language understanding 830 on the speech signal. In the process of sentence analysis 820, in order to perform natural language understanding 830, sentence symbols are processed and the sentence is segmented. Figure 8 , the voice signal “Where does that panda live?” of the user 10 is segmented into [(reference: that) / (subject: animal: panda) / (question: where) / (object: lives in)]. Thereafter, the voice recognition apparatus 100 according to the embodiment may understand the sentence by performing natural language understanding 830 on the analyzed sentence.

[0096] Figure 8 The processing of sentence analysis 820 and natural language understanding 830 shown in FIG. 8 is illustrative only and may be performed by other than Figure 8In addition, the speech recognition device 100 can use AI technology to recognize, apply and process the language of the speaker included in the audio signal. In more detail, the speech recognition device 100 can improve the accuracy of speech recognition by learning (deep learning) the personal characteristics of the user 10 (e.g., intonation, language habits, frequently used words and expressions, etc.) or language understanding of the speech signal (e.g., natural language processing, etc.).

[0097] According to an embodiment, the speech recognition device 100 performs content shortage and / or ambiguity analysis 840 on the speech recognition result. This process is to determine whether the speech recognition device 100 has accurately understood the user input through speech recognition. As a result of analyzing the speech recognition result, when the content contained in the speech signal is insufficient (or ambiguous) for the speech recognition device 100 to accurately understand the user input, the speech recognition device 100 can use the content information to perform natural language understanding result correction 850. That is, the user input recognized by speech recognition can be corrected.

[0098] In more detail, Figure 7 As shown in , the speech recognition device 100 can obtain content information including content story information 710 of the content being executed and content execution information 720. The speech recognition device 100 can supplement or clear the content included in the speech signal through the content information to relatively accurately understand the user 10's speaking intention (e.g., the user input actually intended by the user 10).

[0099] For example, refer to Figure 8 , the voice recognition device 100 can understand the question of the user 10 actually intended by the user 10, which is asking where "the panda in the enclosure at time point 100 / 500 that appears in episode 12 of the program 'Going to the Zoo'" lives, based on the content story information 710 and the content execution information 720 of the content being executed. In addition, the voice recognition device 100 can relatively accurately understand the intention of the user 10 using AI technology. For example, the voice recognition device 100 can relatively accurately understand the user input by logically inferring and predicting the intention of the user 10 based on the content information.

[0100] Fig. 9 is a block diagram illustrating a method for acquiring context information by the voice recognition apparatus 100 according to an embodiment.

[0101] The voice recognition apparatus 100 according to an embodiment may acquire context information 910 related to a situation in which the voice recognition apparatus 100 operates. Fig. 9According to an embodiment, the speech recognition apparatus 100 may acquire context information 910 using a detector (eg, including a detection circuit) 920 configured to detect external information, a user profile 901 , and the like.

[0102] The voice recognition apparatus 100 may use the information detected by the detector 920 to receive and detect various pieces of information for determining the situation in which the voice recognition apparatus 100 operates. For example, the detector 920 may receive or detect the position of the voice recognition apparatus 100, information related to movement, information about a user using the voice recognition apparatus 100, the peripheral environment of the voice recognition apparatus 100, and the like.

[0103] The user profile 901 may refer to, for example, information about a user using the voice recognition apparatus 100, and may be stored in the voice recognition apparatus 100 or received from an external device through a communication interface (not shown). In addition, the voice recognition apparatus 100 may update the user profile 901 based on user information acquired through the detector 920.

[0104] Fig.10 is a block diagram illustrating a method for generating a response by the speech recognition apparatus 100 according to an embodiment.

[0105] Reference Fig.10 According to an embodiment, the voice recognition apparatus 100 may generate a response using the response generation 1010 in response to a user command according to a recognized user input. Here, the response may include an operation of the voice recognition apparatus 100 according to the user input.

[0106] The voice recognition apparatus 100 may generate a response based on content information including content story information 710 and content execution information 720 of content being executed, and may also generate a response based on context information 910 of the voice recognition apparatus 100. In addition, the voice recognition apparatus 100 may generate a response using a database.

[0107] For example, in order to generate a response to the user's question "Where does that panda live?", the voice recognition device 100 may obtain the name, address, etc. of the zoo that raises "the panda in the enclosure that appears at time point 100 / 500 in episode 12 of the program 'Go to the Zoo'". In addition, when the user asks about the story of a drama currently being broadcast on TV, the voice recognition device 100 may generate a response by obtaining only the story after the time point that the user has watched from the content information or the database. The voice recognition device 100 may use the user feature information included in the context information 910 to generate a response suitable for the age limit for teenage users, thereby generating a user-adaptive response.

[0108] In addition, the voice recognition apparatus 100 may use AI technology to generate the best response. For example, the voice recognition apparatus 100 may use a database through knowledge management to generate a response that is closest to the response expected by the user.

[0109] Fig.11 is a flow chart illustrating a method for determining relevance of a response by the speech recognition apparatus 100 according to an embodiment.

[0110] The speech recognition apparatus 100 according to the embodiment may determine the relevance of the generated response and may correct the response when it is determined that the response is inappropriate. When it is determined that the output response is not useful to the user, the response may be corrected even if the response itself is a correct response.

[0111] Reference Fig.11 In operation 1110, the speech recognition apparatus 100 may analyze the universal relevance of the generated response based on the relevance determination database 1101. The relevance determination database 1101 stores data related to criteria that are considered to be appropriate based on universal effectiveness, and the relevance determination database 1101 may be learned and updated using AI technology. The relevance determination database 1101 may be stored in the speech recognition apparatus 100, or may be stored in an external device so that the speech recognition apparatus 100 uses the relevance determination database 1101 by accessing the external device via a communication interface (not shown).

[0112] For example, an answer to a question such as "How to interrupt others?" is not generally effective from a social and ethical perspective, and therefore, it can be analyzed that the answer is inappropriate. In addition, when an answer to a question of a user who is watching a movie may be a spoiler because the answer contains information related to the following synopsis, outputting the answer may inevitably interrupt the user's watching of the movie, and therefore, it can be analyzed that the answer is inappropriate from a general perspective.

[0113] In addition, according to an embodiment, when analyzing the general relevance of the response, the voice recognition apparatus 100 may score the corresponding response according to a preset standard. In operation 1130, the score of the corresponding response may be used to determine the relevance.

[0114] In operation 1120, the speech recognition device 100 may analyze the personal relevance of the generated response based on the user profile 901. As described above, the user profile 901 is information about the user using the speech recognition device 100 and may include data related to the user's personal criteria. The user profile 901 may also be learned and updated using AI technology.

[0115] For example, in the above example, it may be analyzed that it is inappropriate to read spoilers from a general perspective, but for a user who desires to watch a movie after understanding the overall synopsis, it does not matter if the answer includes spoilers.

[0116] In addition, according to an embodiment, when analyzing the personal relevance of the response, the voice recognition apparatus 100 may score the corresponding response according to a preset standard. In operation 1130, the score of the corresponding response may be used to determine the relevance.

[0117] although Fig.11 It is shown that operation 1110 is performed first and operation 1120 is performed subsequently, but operations 1110 and 1120 may be performed sequentially or in parallel, and when operations 1110 and 1120 are performed sequentially, operation 1120 may be performed first and operation 1110 may be performed subsequently.

[0118] In operation 1130, the speech recognition device 100 determines the relevance of the generated answer based on the analysis result of the previous operation. According to an embodiment, the speech recognition device 100 can determine whether the corresponding answer is appropriate based on the score of the answer scored in the universal relevance analysis and the personal relevance analysis. In this case, the speech recognition device 100 can have a preset standard and can score the corresponding standard.

[0119] When it is determined that the generated response is appropriate, the speech recognition device 100 may output the generated response, and when it is determined that the generated response is inappropriate, the speech recognition device 100 may perform operation 1140 to correct the response and output the corrected response. According to an embodiment, the speech recognition device 100 may correct the response in a manner that satisfies a preset standard (i.e., a manner that satisfies a set score).

[0120] Furthermore, when it is determined that the answer is inappropriate or the answer is corrected, the voice recognition apparatus 100 may allow the user to select whether to output the generated answer itself or the corrected answer, and output the answer according to the user's selection.

[0121] The speech recognition device 100 may use AI techniques to generate the best response. For example, the speech recognition device 100 may use a database through knowledge management to determine the relevance of the response.

[0122] Fig.12 is a block diagram illustrating a method for determining an output form of a response by the voice recognition apparatus 100 according to an embodiment.

[0123] The speech recognition device 100 according to the embodiment may determine the output form of the response. In more detail, the speech recognition device 100 may determine the output form 1210 of the response based on the content information including the content story information 710 of the content being executed and the content execution information 720. In addition, the speech recognition device 100 may determine the response form based on the context information 910 of the speech recognition device 100.

[0124] For example, when the generated response is not suitable for output to the public in a public place because the generated response includes violent or innuendo content, the voice recognition device 100 may determine the output form of the response by outputting the response by voice only when the user uses headphones or outputting the image or video in a reduced size when the response is output as an image or video. In addition, when the commentary of the panda's life is being output in the content being executed at the time point of outputting the response or the user is in an environment where the response cannot be correctly heard due to the noise outside the voice recognition device 100, the voice recognition device 100 may output the response as subtitles instead of voice.

[0125] The voice recognition apparatus 100 may use AI technology to determine the output form of the response. For example, the voice recognition apparatus 100 may identify the situation of the voice recognition apparatus 100 operation from the detected external information, and determine a suitable output form according to the identified situation.

[0126] Fig.13 is a block diagram showing a speech recognition apparatus 100 according to an embodiment, and Fig.14 is a block diagram illustrating a voice recognition apparatus 100 according to an embodiment.

[0127] like Fig.13 As shown in FIG. 1 , the speech recognition device 100 according to the embodiment may include a receiver (eg, including a receiving circuit) 1310, a processor (eg, including a processing circuit) 1320, and an output unit (eg, including an output circuit) 1330. However, the speech recognition device 100 according to the embodiment may include a receiver (eg, including a receiving circuit) 1310, a processor (eg, including a processing circuit) 1320, and an output unit (eg, including an output circuit) 1330. Fig.13 The components shown in FIG. 1 and FIG. 2 may be used to implement the speech recognition device 100. For example, Fig.14 As shown in , the speech recognition device 100 according to the embodiment may also include at least one component of a memory 1340, a communication interface (e.g., including a communication interface circuit) 1350, a user input interface (e.g., including a user input circuit) 1360 and a detector (e.g., including a detection circuit) 1370.

[0128] According to an embodiment, the voice recognition device 100 may be included in, for example but not limited to, a non-mobile computing device, a mobile computing device, an electronic control device in a vehicle, a server, or may be connected to at least one of a non-mobile computing device, a mobile computing device, an electronic control device in a vehicle, and a server in a wired or wireless manner, but is not limited thereto.

[0129] The receiver 1310 according to an embodiment may include various receiving circuits for receiving audio signals. For example, the receiver 1310 may directly receive an audio signal by converting an external sound into electrical sound data using a microphone. The receiver 1310 may receive an audio signal transmitted from an external device. Fig.13 The receiver 1310 is shown to be included in the voice recognition apparatus 100 , but according to another embodiment, the receiver 1310 may be included in a separate device and connected to the voice recognition apparatus 100 in a wired or wireless manner.

[0130] The processor 1320 according to an embodiment may include various processing circuits and control general operations of the voice recognition apparatus 100. For example, the processor 1320 may control the receiver 1310 and the output unit 1330. In addition, the processor 1320 may control the operation of the voice recognition apparatus 100 using AI technology.

[0131] The processor 1320 according to an embodiment may analyze a user command based on the content being executed, and provide a suitable response in a suitable form according to the analysis. In more detail, the processor 1320 may perform voice recognition on an audio signal, obtain content information of the content being executed, analyze a user input from a voice recognized by performing voice recognition based on the content information, and generate a response based on the analyzed user input and content information. In addition, the processor 1320 may output the generated response in a suitable form through the output unit 1330.

[0132] The content information may include, for example, but not limited to, story information and execution information of the content being executed. For example, the story information of the content may include information about the content itself, such as an outline of the content, the status of objects included in the content, and the relationship between the objects. In addition, the execution information of the content may include information indicating the execution status of the content, such as the name of the content and the progress of the content. The processor 1320 may acquire the content information by analyzing and processing the content being executed, or acquire the content information from an external device. Therefore, as Fig.14 As shown in , the speech recognition device 100 may further include a communication interface 1350 .

[0133] The processor 1320 according to an embodiment may perform natural language understanding on the recognized voice based on the content information. The processor 1320 may perform natural language understanding on the recognized voice and correct the natural language understanding on the recognized voice based on the content information.

[0134] The processor 1320 according to an embodiment may determine the relevance of the answer and may correct the answer when determining that the answer is inappropriate. In this case, the processor 1320 may determine the universal relevance and / or user relevance of the answer.

[0135] The processor 1320 according to an embodiment may control the detector 1370 to obtain context information related to the situation in which the voice recognition device 100 operates, and generate a response based on the context information. The context information may include information related to the position, movement, peripheral environment of the voice recognition device 100, whether the voice recognition device 100 is connected to another electronic device, characteristics of the user using the voice recognition device 100, etc. In addition, the processor 1320 may analyze the user input based on the context information.

[0136] The processor 1320 according to an embodiment may determine an output form of a response based on content information and / or context information.

[0137] The processor 1320 according to the embodiment may be implemented by hardware and / or software configurations that perform specific functions. For example, the processor 1320 may include at least one of a content information acquirer (not shown), a speech recognizer (not shown), a user command recognizer (not shown), and a response generator (not shown), wherein the content information acquirer (not shown) is configured to acquire content information of the content being executed, the speech recognizer (not shown) is configured to perform speech recognition on the audio signal, the user command recognizer (not shown) is configured to recognize the user command included in the audio signal based on the content information, and the response generator (not shown) is configured to generate a response based on the content information in response to the user command. Each of the aforementioned functional elements may include various processing circuits and / or program elements.

[0138] For example, the functions performed by the processor 1320 according to the embodiment may be implemented by at least one microprocessor or circuit configuration for the corresponding functions. Some or all of the functions performed by the processor 1320 may be implemented by software modules configured to use various programming languages ​​or scripting languages ​​to be executed by the processor 1320. Fig.13 and Fig.14 The voice recognition apparatus 100 is shown to include one processor 1320 , but the voice recognition apparatus 100 is not limited thereto and may include a plurality of processors.

[0139] The output unit 1330 according to an embodiment may include various output circuits and output the result of performing voice recognition on the audio signal. The output unit 1330 may notify the user of the result of performing voice recognition, or may send the result of performing voice recognition to an external device (e.g., a smart phone, a smart TV, a smart watch, a server, etc.).

[0140] In more detail, the output unit 1330 according to the embodiment may output a response corresponding to the result of performing voice recognition. For example, the voice recognition device 100 may output a response to a user's question through the output unit 1330. In addition, the voice recognition device 100 may determine a function of the voice recognition device 100 corresponding to the result of performing voice recognition, and output a screen image for performing the corresponding function through the output unit 1330. Alternatively, the voice recognition device 100 may send a keyword corresponding to the result of performing voice recognition to an external server, receive information related to the sent keyword from the server, and output the received information on the screen through the output unit 1330.

[0141] The output unit 1330 according to an embodiment may output information received from the outside, processed by the processor 1320, or stored in the voice recognition device 100 in the form of at least one of light, sound, image, and vibration. For example, the output unit 1330 may further include at least one of a display configured to output text or an image, a sound output unit configured to output sound, and a vibration motor configured to output vibration.

[0142] like Fig.14 As shown in , the speech recognition apparatus 100 according to the embodiment may further include at least one component of a memory 1340 , a communication interface 1350 , a user input interface 1360 , and a detector 1370 .

[0143] According to an embodiment, the memory 1340 may include at least one type of storage medium among a flash memory, a hard disk memory, a multimedia card micro memory, a card-type memory (for example, a secure digital (SD) memory, an extreme digital (XD) memory, etc.), a random access memory (RAM), a static RAM (SRAM), a read-only memory (ROM), an electrically erasable programmable ROM (EEPROM), a programmable ROM (PROM), a magnetic memory, a disk, and an optical disk.

[0144] The user input interface 1360 according to an embodiment may include various user input interface circuits, and receive user input for controlling the voice recognition apparatus 100. The user input interface 1360 may include a user input device including an input circuit, wherein the input circuit includes, for example but not limited to, a touch panel configured to receive a user's touch, a button configured to receive a user's pressing operation, a wheel configured to receive a user's rotation operation, a keyboard, a dome switch, etc., but is not limited thereto.

[0145] The communication interface 1350 according to the embodiment may include various communication interface circuits and communicate with an external electronic device or server through wired or wireless communication. The communication interface 1350 may acquire content information, context information of the speech recognition apparatus 100, etc. For example, the communication interface 1350 may communicate with an external device having a database to acquire content information, context information of the speech recognition apparatus 100, etc.

[0146] The communication interface 1350 according to an embodiment may include, for example but not limited to, a near field communication module, a wired communication module, a mobile communication module, a broadcast receiving module, and the like.

[0147] The detector 1370 may include various detection circuits, such as, for example but not limited to, one or more sensors, and receive and detect various pieces of information for determining the situation in which the voice recognition device 100 operates. For example, the detector 1370 may receive or detect the position of the voice recognition device 100, information related to movement, information about a user using the voice recognition device 100, the peripheral environment of the voice recognition device 100, etc.

[0148] For example, the detector 1370 may include, for example but not limited to, at least one or a combination of illumination sensors, biometric sensors, tilt sensors, position sensors, proximity sensors, geomagnetic sensors, gyroscope sensors, temperature / humidity sensors, infrared sensors and / or speed / acceleration sensors, etc.

[0149] According to an embodiment, voice recognition may be performed based on the content being executed, and a suitable response according to the voice recognition may be provided in a suitable form.

[0150] The speech recognition method or the speech recognition apparatus 100 according to the embodiment has been described. Figures 15 to 20 An example of using the speech recognition method or the speech recognition apparatus 100 according to the embodiment is described.

[0151] Fig.15 is a diagram showing an example of outputting a response that varies depending on the content being executed.

[0152] like Fig.15 As shown in , the voice recognition apparatus 100 can output a response that varies according to the content being executed for the voice signal of the user 10. Fig.15 , for example, when the user 10 asks the question “What is 11?”, the speech recognition apparatus 100 may analyze and understand the meaning of the question, wherein the meaning varies depending on the content being executed.

[0153] When the user 10 is watching a baseball game, the voice recognition device 100 may obtain information about a baseball player with a back number of 11, and output a response "the baseball player with a back number of 11 in baseball team A is X". When the user 10 is watching a football game, the voice recognition device 100 may obtain information about a football player with a back number of 11, and output a response "the football player with a back number of 11 in football team B is Y".

[0154] When the user 10 is watching a channel guide or changing channels, the voice recognition device 100 can obtain information about channel number 11 and output a response "Channel number 11 is broadcasting station C, and program D is currently being broadcast." In addition, when the user 10 is watching a performance of a certain singer in a music video or music program, the voice recognition device 100 can obtain information about track number 11 of the corresponding album of the singer and output a response "Track number 11 is E. Do you want to listen to it?".

[0155] According to an embodiment, voice recognition may be performed based on the content being executed, and a suitable response according to the voice recognition may be provided.

[0156] Fig.16 is a diagram showing an example in which the voice recognition apparatus 100 outputs a response that varies according to context information.

[0157] Reference Fig.16 , the user 10 asks for a summary of the content being executed. Usually, since the user 10 will know the summary of the part that the user 10 has already watched, the user 10's intention is likely to ask for the content of the part that the user 10 has not yet watched. In this case, the existing voice recognition device consistently understands the user's question "I'm a little late. What's the story?" and outputs a general summary of the content being executed as a response "In the summer of 1998, the protagonist 'X'...". Therefore, the user 10 should again command "No, only from the part I have watched until now".

[0158] Compared with the existing speech recognition device, the speech recognition device 100 according to the embodiment can use the user feature information included in the context information 910 of the speech recognition device 100 to only obtain the content after the time point that the user 10 has watched, and output a response suitable for the intention of the user 10 "The wedding of 'X' was held."

[0159] According to an embodiment, the voice recognition apparatus 100 may quickly and conveniently output a response suitable for the intention of the user 10 .

[0160] Fig.17 is a diagram showing another example of outputting a response that varies depending on the content being executed.

[0161] Reference Fig.17 , in the case of dialogue 1 and dialogue 2, the user 10 asks the same question while watching different pieces of content. In the case of dialogue 1, the user 10 is watching a rare animal documentary, and in the case of dialogue 2, the user 10 is watching a zoo-related program.

[0162] Even when the voice recognition device 100 receives the same question from the user 10, the voice recognition device 100 according to the embodiment can also output a response that changes according to the content being executed. In the case of dialogue 1 in which the user 10 is watching a documentary about rare animals, the voice recognition device 100 can obtain information about the main habitats of pandas based on the content of the documentary about rare animals and output the response "pandas are generally distributed in China". In addition, in the case of dialogue 2 in which the user 10 is watching a zoo-related program, the voice recognition device 100 can obtain information about a zoo that raises pandas broadcast at the time point the user is watching, and output the response "it lives in Y Zoo" based on the content of the zoo-related program.

[0163] According to an embodiment, the voice recognition apparatus 100 may output a response suitable for the content being executed.

[0164] Fig.18 is a diagram illustrating an example of determining the relevance of a response.

[0165] Reference Fig.18 , in the case where the user 10 is watching a suspense movie, the user 10 asks questions about the characters. Generally, when watching a suspense movie, the user 10 does not want to watch the suspense movie after knowing the ending in advance. Therefore, when the answer may be a spoiler because the answer contains information about the ending, the voice recognition device 100 may disturb the user 10 from watching the movie by outputting the answer. In this case, the voice recognition device 100 may appropriately correct the answer, or determine whether to output the answer after checking the intention of the user 10.

[0166] In the case of Dialogue 3, the existing voice recognition device consistently understands the question of the user 10 "Why did that man come suddenly?" and outputs the answer "Because he is a criminal" including information about the ending, regardless of the genre and story of the movie.

[0167] Compared with existing voice recognition devices, in the case of dialogue 4, the voice recognition device 100 according to the embodiment can understand based on the content information that the answer may be a spoiler because the answer contains information about the ending, and output a relatively appropriate answer by verifying with the user 10 whether to output the answer via "This may be a spoiler question, do you want to hear it?"

[0168] According to an embodiment, the voice recognition apparatus 100 may determine relevance of responses and output an optimal response.

[0169] Fig.19 : is a diagram showing an example form of a response output according to the embodiment.

[0170] Reference Fig.19 , in the case where the user is watching a movie, the user 10 asks a question about the name of a vehicle that appears in the movie. The voice recognition apparatus 100 may output a response in a suitable form in consideration of content information, context information of the voice recognition apparatus 100, and the like. For example, when a response is output by voice 1910 at a point in time when a character in a movie is speaking, the output response overlaps with the dialogue in the movie, and thus the user 10 cannot hear the response correctly, and the response may disturb watching the movie. Therefore, in this case, the response may be output as subtitles 1920 instead of voice.

[0171] According to an embodiment, the voice recognition apparatus 100 may output a response in a suitable form according to the content.

[0172] Fig. 20 is a diagram illustrating another example in which the voice recognition apparatus 100 according to the embodiment outputs a response that varies according to context information.

[0173] Reference Fig. 20 , the speech recognition apparatus 100 may use the user characteristic information included in the context information 910 of the speech recognition apparatus 100 to generate a user adaptive response.

[0174] For example, when the user 10 is watching a football game, the voice recognition device 100 may obtain information about the team supported by the user 10, the preferred voice tone (preference according to gender), the preferred commentator, etc. When the user 10 supports team A and prefers a female voice tone, the voice recognition device 100 may find out the commentary that tends to support team A according to the user characteristic information and output the commentary in a female voice tone. In addition, when the preferred commentator comments on the football game, the voice recognition device 100 may find out and output the commentary of the preferred commentator.

[0175] According to an embodiment, the voice recognition apparatus 100 may output a response in a suitable form according to user characteristics.

[0176] The above-described embodiments can be written as computer executable programs and can be implemented in general-purpose digital computers that use non-transitory computer-readable recording media to execute programs. In addition, the structure of the data used in the above-described embodiments can be recorded in a non-transitory computer-readable recording medium in various ways. In addition, the above-described embodiments can be implemented in the form of a recording medium (such as a computer-executable program module) including computer executable instructions. For example, the method implemented using a software module or an algorithm can be stored in a non-transitory computer-readable recording medium in the form of a computer-readable code or program instruction executable in a computer system.

[0177] Non-transitory computer-readable media may be any recording medium that can be accessed by a computer system, and include all types of volatile and non-volatile media and separate and non-separate media. Non-transitory computer-readable media may include magnetic storage media (e.g., ROM, floppy disks, hard disks, etc.) and optical reading media (e.g., storage media such as compact disk ROM (CD-ROM) or digital versatile disk (DVD)), but are not limited thereto. In addition, non-transitory computer-readable media may include computer storage media and communication media.

[0178] A plurality of non-transitory computer-readable recording media may also be distributed over a network coupled to computer systems, and data (eg, program instructions and codes) stored in the distributed recording media may be executed by at least one computer system.

[0179] The specific execution steps described in this disclosure are illustrative and do not limit the scope of this disclosure. For the sake of brevity of this disclosure, existing electronic configurations, control systems, software and other functional aspects of the disclosed system may be omitted.

[0180] It should be understood that the various exemplary embodiments described herein should be considered in an illustrative sense only and not for purposes of limitation. Descriptions of features or aspects in each embodiment should typically be considered as available for other similar features or aspects in other embodiments.

[0181] Although various example embodiments have been described with reference to the accompanying drawings, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope, for example but not limited to, as defined in the claims.

Claims

1. A method performed by a speech recognition device, the method comprising: receiving an audio signal and performing speech recognition on the audio signal; Obtaining content information of content being executed on the speech recognition device; Obtaining contextual information about the circumstances under which the speech recognition device operates; analyzing a user input from a speech recognized by performing speech recognition based on the content information and the context information; generating a response based on the analyzed user input, the content information, and the context information; and outputting the response in a suitable form based on the content information and the context information, The step of obtaining context information of the operation of the speech recognition device includes: obtaining the context information using a detector configured to detect external information and a user profile, wherein the user profile includes information about a user using a speech recognition device, wherein the user profile is updated based on user information obtained by the detector, The step of generating the response includes: determining the general relevance and / or user relevance of the response; and When it is determined that the response is inappropriate, correcting the response, The step of determining the general relevance of the response comprises: determining whether the response is considered valid from a social and ethical perspective, The step of determining the user relevance of the answer comprises: determining based on a user profile comprising data related to personal criteria of the user.

2. The method of claim 1, wherein: The step of analyzing a user input from a voice recognized by performing voice recognition based on the content information and the context information includes performing natural language understanding on the recognized voice based on the content information.

3. The method of claim 2, wherein: The step of performing natural language understanding on the recognized speech based on the content information includes: performing natural language understanding on the recognized speech; and Based on the content information, natural language understanding of the recognized speech is corrected.

4. The method of claim 1, wherein: The content information includes content story information related to a story of the content being executed and / or content execution information related to execution of the content being executed.

5. The method of claim 1, wherein: The step of outputting the response in a suitable form based on the content information and the context information includes: determining an output form of the response based on the content information and / or the context information.

6. The method of claim 1, wherein: The context information includes at least one of the following information: a location, a movement, a peripheral environment of the voice recognition device, whether the voice recognition device is connected to another electronic device, and a feature of a user of the voice recognition device.

7. A speech recognition device, comprising: a receiver including a receiving circuit configured to receive an audio signal; At least one processor configured to: performing speech recognition on the audio signal, Get the content information of the content being executed on the speech recognition device. Get contextual information about the operation of the speech recognition device. acquiring the context information using a detector configured to detect external information and a user profile, wherein the user profile includes information about a user using a speech recognition device, wherein the user profile is updated based on the user information obtained by the detector, analyzing a user input from a voice recognized by performing voice recognition based on the content information and the context information, generating a response based on the analyzed user input, the content information and the context information, determining a general relevance and / or a user relevance of the response, Determine whether the response is considered valid from a social and ethical perspective, The determination is made based on a user profile including data related to the user's personal criteria, When it is determined that the response is inappropriate, correcting the response; and The output unit includes an output circuit configured to output the response in a suitable form based on the content information and the context information.

8. The speech recognition device according to claim 7, wherein: The at least one processor is further configured to: Natural language understanding is performed on the recognized speech based on the content information.

9. The speech recognition device according to claim 7, wherein: The at least one processor is further configured to: Natural language understanding of the recognized speech is performed, and the natural language understanding of the recognized speech is corrected based on the content information.

Images