Electronic device for speech recognition and control method therefor

Abstract

This disclosed electronic device comprises: a memory storing at least one instruction; and at least one processor including a circuit device, wherein the at least one instruction is individually or collectively executed by the at least one processor to: when at least one security word is included in an acquired speech input, perform a function corresponding to speech recognition for the speech input and control text corresponding to the speech input to be output; and when at least one security word is not included in the acquired speech input, control the text corresponding to the speech input to be output on the basis of a function corresponding to automatic speech recognition for the speech input, performed by a server.

Description

Electronic device for voice recognition and method of operation thereof

[0001] One embodiment of the present disclosure relates to an electronic device and a method of operating the same, and more specifically, to an electronic device that recognizes voice input and a method for performing the same.

[0002] Automatic Speech Recognition is a technology that converts human speech into text. It is utilized in various electronic devices such as smartphones, air conditioners, refrigerators, TVs, and AI speakers. First, the electronic device acquires human speech as input and uses a pre-trained speech recognition model to recognize the input speech and convert it into text. The device then obtains the converted text as the final output.

[0003] Recently, deep neural network (DNN) algorithms have been used in various machine learning fields, leading to performance improvements. Significant performance enhancements have also been achieved in the field of speech recognition using neural networks, and Automatic Speech Recognition Models (ASMRs) are currently being researched. As AI systems improve in recognition rates and become capable of understanding user preferences more accurately with continued use, existing rule-based smart systems are gradually being replaced by deep learning-based AI systems.

[0004] The present invention aims to provide an electronic device that recognizes voice input according to a disclosed embodiment and outputs text corresponding to the voice input, and a method of operation thereof.

[0005] An electronic device according to one embodiment of the present disclosure may include at least one processor comprising a memory for storing at least one instruction and a circuit device. By executing at least one instruction individually or collectively by at least one processor, the electronic device may be controlled to perform a function corresponding to voice recognition for the voice input and output text corresponding to the voice input if the acquired voice input contains at least one security word. By executing at least one instruction individually or collectively by at least one processor, the electronic device may be controlled to output text corresponding to the voice input based on a function corresponding to automatic voice recognition performed on a server for the voice input if the acquired voice input does not contain at least one security word.

[0006] A method of operation of an electronic device according to one embodiment of the present disclosure may include, if the acquired voice input contains at least one security word, an operation of controlling to perform a function corresponding to voice recognition for the voice input and output text corresponding to the voice input. The method may include, if the acquired voice input does not contain the at least one security word, an operation of controlling to output text corresponding to the voice input based on a function corresponding to automatic voice recognition performed on a server for the voice input.

[0007] In a computer-readable recording medium having at least one instruction recorded thereon according to one embodiment of the present disclosure, the at least one instruction may be configured to be executed individually or collectively by at least one processor, or to perform the method described below.

[0008] The present invention can be easily understood from the combination of the following detailed description and the accompanying drawings, where reference numerals denote structural elements.

[0009] FIG. 1 is a schematic diagram of a voice recognition system according to one embodiment of the present disclosure.

[0010] FIG. 2 illustrates an example of a system including an electronic device and a server according to one embodiment of the present disclosure.

[0011] FIG. 3 is a block diagram illustrating a module in which an electronic device according to one embodiment of the present disclosure acquires voice input and outputs text.

[0012] FIG. 4 is a flowchart of a method in which an electronic device according to one embodiment of the present disclosure acquires voice input and outputs text.

[0013] FIG. 5 is a flowchart illustrating a detailed operation for identifying whether at least one security word is included in a voice input included in operation 420 according to one embodiment of the present disclosure.

[0014] FIG. 6 is a flowchart for explaining a detailed operation for identifying whether at least one security word is included in the voice input included in operation 420 when a user input corresponding to the modification of a security word is obtained according to one embodiment of the present disclosure.

[0015] FIG. 7 is a reference diagram for showing types of security words according to one embodiment of the present disclosure.

[0016] FIG. 8 is a schematic diagram illustrating a method for outputting text corresponding to a voice input based on a function corresponding to voice recognition according to one embodiment of the present disclosure.

[0017] FIG. 9 is a reference diagram for illustrating a word correction table according to one embodiment of the present disclosure.

[0018] FIG. 10 is a flowchart illustrating a method for training a function corresponding to speech recognition according to one embodiment of the present disclosure.

[0019] FIG. 11a is a flowchart illustrating an operation for training a function corresponding to voice recognition included in operation 1020 according to one embodiment of the present disclosure.

[0020] FIG. 11b is a reference diagram for illustrating an operation of training an acoustic model among the functions corresponding to speech recognition included in operation 1020 according to one embodiment of the present disclosure.

[0021] FIG. 11c is a reference diagram for illustrating an operation of training a language model among the functions corresponding to speech recognition included in operation 1020 according to one embodiment of the present disclosure.

[0022] FIG. 12 is a flowchart illustrating an operation to update a word modification table according to one embodiment of the present disclosure.

[0023] FIG. 13a is a flowchart for explaining in detail the operation of updating a word modification table and the operation of obtaining result text based on user input included in operation 1200 according to one embodiment of the present disclosure.

[0024] FIG. 13b is a reference diagram for explaining in detail the operation of updating a word modification table based on user input and the operation 1300 of obtaining result text according to one embodiment of the present disclosure.

[0025] FIG. 14 is a flowchart illustrating the operation of updating a word modification table by comparing it with the contents of an existing word modification table according to one embodiment of the present disclosure.

[0026] FIG. 15 is a block diagram illustrating the configuration of an electronic device according to one embodiment of the present disclosure.

[0027] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments.

[0028] In relation to the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of the noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise.

[0029] In this document, each of the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "at least one of A, B, or C" may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof.

[0030] The term “and / or” includes a combination of multiple related described components or any of the multiple related described components.

[0031] Terms such as "first," "second," or "first" or "second" may be used simply to distinguish a component from another corresponding component and do not limit the components in other aspects (e.g., importance or order).

[0032] Where any (e.g., 1st) component is referred to as "coupled" or "connected" to another (e.g., 2nd) component, with or without the terms "functionally" or "communicationly," it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0033] Terms such as “include” or “have” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in this document, and do not preclude the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0034] When it is said that one component is “connected,” “combined,” “supported,” or “in contact” with another component, this includes not only cases where the components are directly connected, combined, supported, or in contact, but also cases where they are indirectly connected, combined, supported, or in contact through a third component.

[0035] When it is said that a component is located “on” another component, this includes not only cases where one component is in contact with the other, but also cases where another component exists between the two components.

[0036] It should be understood that the blocks in each flowchart and combinations of flowcharts can be executed by one or more computer programs containing computer-executable instructions. One or more computer programs may be stored all in a single memory or may be partitioned and stored in multiple different memories.

[0037] One embodiment of the present disclosure may be represented by functional block configurations and various processing steps. Some or all of these functional blocks may be implemented by various numbers of hardware and / or software configurations that execute specific functions. For example, the functional blocks of the present disclosure may be implemented by one or more microprocessors or by circuit configurations for a specific function. Additionally, for example, the functional blocks of the present disclosure may be implemented in various programming or scripting languages. The functional blocks may be implemented as algorithms executed on one or more processors. Furthermore, the present disclosure may employ prior art for electronic configuration, signal processing, and / or data processing, etc.

[0038] All functions or operations, including functions related to artificial intelligence according to the present disclosure, are operated through a processor and memory. The processor may be composed of one or more processors. A single processor or a combination of processors may include circuitry that performs processing, such as an AP (Application Processor), CP (Communication Processor), GPU (Graphical Processing Unit), NPU (Neural Processing Unit), MPU (Microprocessor Unit), SoC (System on Chip), IC (Integrated Chip), etc.

[0039] In one embodiment of the present disclosure, security words may include proper nouns such as specific company names, names of specific parts, specific project names, specific product names, etc. Security words may represent words commonly used in situations requiring security as a result of learning. For example, security words may include words related to personal information, such as resident registration numbers, addresses, mobile phone numbers, etc.

[0040] According to one embodiment of the present disclosure, security words may include words obtained through an artificial intelligence model (e.g., a generative artificial intelligence model). For example, even if a specific word is not a proper noun, it may be a word frequently used in a meeting situation regarding a specific part where security is required. In this case, the electronic device (100) may add or include the specific word as a security word through the learning of the artificial intelligence model.

[0041] The present disclosure will be described in detail below with reference to the attached drawings.

[0042] FIG. 1 is a schematic diagram of a voice recognition system according to one embodiment of the present disclosure.

[0043] Referring to FIG. 1, a voice recognition system according to one embodiment of the present disclosure may include an electronic device (100) and a server (2000). Specifically, the electronic device (100) may be an electronic device that recognizes an input voice input and obtains text corresponding to the voice input.

[0044] The electronic device (100) may include a function (320) corresponding to voice recognition. The server (2000) may include a function (2100) corresponding to automatic voice recognition. The electronic device (100) may recognize voice input (10) based on the function (320) corresponding to voice recognition and output text (330) corresponding to the voice input. The electronic device (100) may modify the text generated using the function corresponding to voice recognition. The server (2000) may receive voice input (10) from the electronic device (100). The server (2000) may recognize the received voice input (10) based on the function (2100) corresponding to automatic voice recognition and obtain text corresponding to the voice input. The server (2000) may transmit the obtained text to the electronic device (100). The electronic device (100) may be controlled to output text corresponding to the voice input based on the function corresponding to automatic voice recognition performed by the server.

[0045] The function corresponding to speech recognition may include an Automatic Speech Recognition (ASR) model. The ASR model is a speech recognition model that recognizes speech using an integrated neural network. The ASR model can output text from the user's voice input. The ASR model may be, for example, an artificial intelligence model that includes an acoustic model, a pronunciation dictionary, and a language model. Alternatively, the ASR model may be an end-to-end speech recognition model having a structure that includes an integrated neural network without separately including, for example, an acoustic model, a pronunciation dictionary, and a language model. By utilizing an integrated neural network, the end-to-end ASR model can convert speech into text without the process of converting phonemes into text after recognizing phonemes from speech.

[0046] Text may contain at least one character. A character may refer to a symbol used to represent and write human language in a visible form. For example, a character may include Hangul, the alphabet, Hanja, numbers, phonetic symbols, punctuation marks, and other symbols. Additionally, for example, text may contain a string of characters. A string of characters may refer to a sequence of characters. For example, text may contain at least one grapheme. A grapheme may be the smallest unit representing a sound, consisting of at least one character. For example, in an alphabetic writing system, a single character may be a grapheme, and a string of characters may refer to an array of graphemes. For example, text may contain a morpheme or a word. A morpheme is the smallest unit of meaning, consisting of at least one grapheme. A word is the basic unit of language, consisting of at least one morpheme, that can be used independently or expresses a grammatical function.

[0047] The electronic device (100) can transmit and receive data to and from a server (2000) via a network for voice recognition. The network includes a Local Area Network (LAN), a Wide Area Network (WAN), a Value Added Network (VAN), a mobile radio communication network, a satellite communication network, and combinations thereof, and is a data communication network in a comprehensive sense that enables each network constituent entity shown in FIG. 1 to communicate smoothly with each other, and may include wired internet, wireless internet, and mobile wireless communication networks.

[0048] An electronic device (100) can acquire a voice input (10). The electronic device (100) can identify (12) whether the voice input (10) contains a security word. For example, the electronic device (100) can recognize a pattern of the voice input (10). The electronic device (100) can identify whether the voice input (10) contains a voice pattern corresponding to a security word. For example, the electronic device (100) can recognize the voice input (10) and identify whether it contains a security word.

[0049] Depending on whether the voice input (10) contains a security word, the electronic device (100) can determine whether to perform voice recognition based on a voice recognition function (320) included in the electronic device (100) or to perform voice recognition based on an automatic voice recognition function (2100) included in the server (2000).

[0050] The electronic device (100) can perform a function (320) corresponding to voice recognition included in the electronic device (100) for the voice input (10) if the voice input (10) contains at least one security word. The electronic device (100) can process the voice input (14) containing the security word in the function (320) corresponding to voice recognition. By processing the voice input (14) containing the security word in the function (320) corresponding to voice recognition, the electronic device (100) can obtain text corresponding to the voice input (14) containing the security word. The electronic device (100) can be controlled to output text corresponding to the voice input.

[0051] The electronic device (100) may provide the voice input (10) to the server (2000) to perform voice recognition using a function (2100) corresponding to automatic voice recognition included in the server (2000) if the voice input (10) does not contain at least one security word. The server (2000) may be a server for voice input.

[0052] The electronic device (100) may provide a server (2000) to process a voice input (15) that does not contain a security word in a function (2100) that corresponds to automatic voice recognition. The electronic device (100) may transmit the voice input (15) that does not contain a security word to a function (2100) that corresponds to automatic voice recognition. The electronic device (100) may obtain text corresponding to the voice input (15) that does not contain a security word from the server (2000). The electronic device (100) may output text (330) corresponding to the voice input (15). The electronic device (100) may control the output of text (330) corresponding to the voice input based on the function that corresponds to automatic voice recognition performed by the server (2000) for the voice input.

[0053] The electronic device (100) can automatically detect a security word in the voice input (10). When a security word is detected in the voice input (10), the electronic device (100) can process it using a voice recognition corresponding function (320) included in the electronic device (100), thereby preventing the voice input (10) from being transmitted to the server (2000) in an environment requiring security. Through this, the electronic device (100) can minimize the risk of the voice input (10) containing sensitive information being leaked through an external network. By processing the voice input (10) using the voice recognition corresponding function (320), the electronic device (100) can operate even when there is no connection to the server (2000) or when the connection is unstable. The electronic device (100) can improve the user experience by accurately processing sensitive voice data even in an environment where the network connection is unstable or non-existent.

[0054] The electronic device (100) can train a function (320) that corresponds to voice recognition. For example, the electronic device (100) can train a function (320) that corresponds to voice recognition on its own without a connection to the server (2000). For example, the electronic device (100) can train a function (320) that corresponds to voice recognition based on the history of the user modifying text (330). The electronic device (100) can improve the accuracy of voice recognition by training a function (320) that corresponds to voice recognition. By training a function (320) that corresponds to voice recognition, the electronic device (100) can provide a customized on-device automatic voice recognition service suitable for a specific device or user. The electronic device (100) can train a function (320) that corresponds to voice recognition based on a security word by training a function (320) that corresponds to voice recognition without a connection to the server (2000). Therefore, the electronic device (100) can improve the accuracy of voice recognition while preventing the security word from being uploaded to the server (2000).

[0055] The electronic device (100) can provide various voice assistant services based on acquired text. The voice assistant service may be a service that provides a conversation with the user. The voice assistant service may provide response messages to the user as if a person were having a direct conversation with the user, taking into account the user's situation, the device's situation, etc. Additionally, the voice assistant service may appropriately generate and provide information needed by the user, such as a personal assistant to the user. The voice assistant service may be linked with various services, such as, for example, broadcasting services, content sharing services, content provision services, power management services, game provision services, chat services, document creation services, search services, call services, photo taking services, transportation recommendation services, and video playback services, to provide information or functions needed by the user.

[0056] Hereinafter, we will examine in detail a method for controlling an electronic device (100) according to one embodiment of the present disclosure to perform a function corresponding to voice recognition for a voice input (10) and output text (330) corresponding to the voice input when at least one security word is included in the voice input obtained by the electronic device (100), and a method for training the electronic device (100) to perform a function (320) corresponding to voice recognition.

[0057] FIG. 2 illustrates an example of a system including an electronic device and a server according to one embodiment of the present disclosure.

[0058] According to one embodiment of the present disclosure, an electronic device (100) is a device capable of displaying an image or data upon a user's request and may include a memory (110), a processor (120), and a communication interface (130).

[0059] The electronic device (100) can transmit and receive data through a server (2000) and a network (20) for voice recognition. The network (20) includes a Local Area Network (LAN), a Wide Area Network (WAN), a Value Added Network (VAN), a mobile radio communication network, a satellite communication network, and combinations thereof, and is a data communication network in a comprehensive sense that enables each network constituent entity shown in FIG. 2 to communicate smoothly with each other, and may include wired internet, wireless internet, and mobile wireless communication networks.

[0060] The electronic device (100) can be implemented in various forms. The electronic device (100) can be any type of device that performs functions including a processor and memory. The electronic device (100) can be a stationary or portable device. For example, the electronic device (100) can represent a device equipped with a display capable of displaying image content, video content, game content, graphic content, etc. The electronic device (100) can include various types of electronic devices capable of receiving and outputting content, such as televisions like network TV, smart TV, internet TV, web TV, and IPTV; computers like desktops, laptops, and tablets; smartphones, cellular phones; game players, music players, video players; medical equipment; e-book readers; digital broadcasting terminals; navigation systems; kiosks; MP3 players; digital cameras; and various smart devices such as home appliances. The electronic device (100) can be referred to as a display device in terms of displaying content, and may also be referred to as a content providing device, a computing device, etc. Additionally, the electronic device (100) may be a wearable device such as a watch, glasses, a hair band, and a ring equipped with communication and data processing functions. However, it is not limited thereto, and the electronic device (100) may include any type of device capable of voice recognition.

[0061] The memory (110) can store a program for processing and controlling the processor (120), and can store data that is input to or output from the electronic device (100). Additionally, the memory (110) can store data necessary for the operation of the electronic device (100).

[0062] The memory (110) may include at least one type of storage medium among flash memory type, hard disk type, multimedia card micro type, card type memory (e.g., SD or XD memory, etc.), RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk, and optical disk.

[0063] The memory (110) may not exist separately but may be configured to be included in the processor (120). The memory (110) may be composed of volatile memory, non-volatile memory, or a combination of volatile and non-volatile memory. The memory (110) may store a program or at least one instruction for performing operations according to the embodiments described below. The memory (110) may provide stored data to the processor (120) upon the request of the processor (120).

[0064] The processor (120) controls the overall operation of the electronic device (100). The processor (120) is configured to control a series of processes to enable the electronic device (100) to operate according to the embodiments described below, and may be composed of one or more processors. The one or more processors included in the processor (120) may be circuitry such as a System on Chip (SoC) or an Integrated Circuit (IC). The one or more processors included in the processor (120) may be a general-purpose processor such as a CPU (Central Processing Unit), MPU (Micro Processor Unit), AP (Application Processor), or DSP (Digital Signal Processor); a graphics-dedicated processor such as a GPU (Graphic Processing Unit) or VPU (Vision Processing Unit); an artificial intelligence-dedicated processor such as an NPU (Neural Processing Unit); or a communication-dedicated processor such as a CP (Communication Processor). If one or more processors included in the processor (120) are artificial intelligence dedicated processors, the artificial intelligence dedicated processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0065] The processor (120) can write data to the memory (110) or read data stored in the memory (110), and in particular, can process data according to a predefined operation rule or artificial intelligence model by executing a program or at least one instruction stored in the memory (110). The processor (120) can perform operations described in subsequent embodiments, and operations described as being performed by the electronic device (100) in subsequent embodiments may be considered to be performed by the processor (120) unless otherwise specified.

[0066] For example, the processor (120) may perform the function of the electronic device (100) described in the present disclosure by individually or collectively executing at least one instruction stored in the memory (110). Alternatively, according to one embodiment, the electronic device (100) may perform the function described in the present disclosure by individually or collectively executing at least one instruction stored in the memory (110) by the processor (120). Accordingly, the processor (120) may perform the operations described in subsequent embodiments, and the operations described as being performed by the electronic device (100) or detailed components included in the electronic device (100) in subsequent embodiments may be considered to be performed by the processor (120) unless otherwise specified.

[0067] The communication interface (130) can communicate with at least one electronic device. The communication interface (130) can transmit and receive information for voice recognition and voice assistant services with the server (2000) and an external device (not shown). Here, 'communication' may mean the operation of transmitting and / or receiving data, signals, requests, and / or commands, etc. The communication interface (130) can perform wired or wireless communication with at least one electronic device. The electronic device (100) can communicate with the server (2000) through the communication interface (130).

[0068] For example, the communication interface (130) may include at least one of a communication module, a communication circuit, a communication device, an input / output port, and an input / output plug for performing wired or wireless communication with at least one electronic device. For example, the communication interface (130) may include at least one wireless communication module, a wireless communication circuit, or a wireless communication device for performing wireless communication with at least one electronic device.

[0069] For example, the communication interface (130) may include a short-range communication module, such as an IR (infrared) communication module, capable of receiving control commands from a remote controller located at a short distance. In this case, the communication interface (130) may receive control signals from the remote controller.

[0070] For example, the communication interface (130) may include at least one communication module that performs communication according to wireless communication standards such as Bluetooth, Wi-Fi, BLE (Bluetooth Low Energy), NFC (Near Field Communication), RFID (Radio Frequency Identification), Wi-Fi Direct, UWB, or Zigbee. Alternatively, the communication interface (130) may further include a communication module that performs communication with a server to support long-distance communication according to long-distance communication standards. For example, the communication interface (130) may include a communication module that performs communication through a network for internet communication. Additionally, the communication interface (130) may include a communication module that performs communication through a communication network according to communication standards such as 3G, 4G, 5G and / or 6G.

[0071] In one embodiment, the electronic device (100) can obtain voice input by executing at least one instruction stored in memory (110) individually or collectively by the processor (120).

[0072] In one embodiment, the electronic device (100) can identify whether the voice input contains at least one security word by individually or collectively executing at least one instruction stored in memory (110) by the processor (120). In one embodiment, the electronic device (100) can obtain a similarity between the voice input and the at least one security word by individually or collectively executing at least one instruction stored in memory (110) by the processor (120). In one embodiment, the electronic device (100) can identify that the voice input contains at least one security word if the similarity is greater than or equal to a threshold similarity by individually or collectively executing at least one instruction stored in memory (110) by the processor (120).

[0073] In one embodiment, the electronic device (100) can obtain user input corresponding to the modification of the at least one security word by individually or collectively executing at least one instruction stored in memory (110) by the processor (120). In one embodiment, the electronic device (100) can identify whether the at least one security word reflecting the user input is included in the voice input by individually or collectively executing at least one instruction stored in memory (110) by the processor (120).

[0074] In one embodiment, the electronic device (100) can select whether to perform voice recognition based on a voice recognition function included in the electronic device or to perform voice recognition based on an automatic voice recognition engine included in the server, depending on whether the voice input contains at least one security word, by executing at least one instruction stored in memory (110) individually or collectively by the processor (120).

[0075] In one embodiment, the electronic device (100) can control the execution of at least one instruction stored in memory (110) by the processor (120) individually or collectively, so that if the acquired voice input contains at least one security word, it performs a function corresponding to voice recognition for the voice input and outputs text corresponding to the voice input.

[0076] In one embodiment, the electronic device (100) can control the output of text corresponding to the voice input based on a function corresponding to automatic speech recognition performed on a server for the voice input, by individually or collectively executing at least one instruction stored in memory (110) by a processor (120), if the acquired voice input does not contain the at least one security word. In one embodiment, the electronic device (100) can transmit the voice input to a server for speech recognition by individually or collectively executing at least one instruction stored in memory (110) by a processor (120). In one embodiment, the electronic device (100) can receive text corresponding to the voice input from a server for speech recognition by individually or collectively executing at least one instruction stored in memory (110) by a processor (120).

[0077] In one embodiment, the electronic device (100) can train a function corresponding to speech recognition by individually or collectively executing at least one instruction stored in memory (110) by the processor (120). For example, the electronic device (100) can train a function corresponding to speech recognition by individually or collectively executing at least one instruction stored in memory (110) by the processor (120) when, based on a word correction table included in the electronic device, a word to be corrected included in the text is changed to a replacement word more than a threshold number of times.

[0078] For example, the electronic device (100) can update the probability of obtaining the replacement word from the voice input based on the characteristics of at least one voice input containing the word to be modified by executing at least one instruction stored in memory (110) individually or collectively by the processor (120).

[0079] For example, the electronic device (100) can update the probability of obtaining the replacement word from any text based on the combination order of at least one word included in the modified text in which the modified word included in the text is changed to the replacement word by executing at least one instruction stored in memory (110) by the processor (120) individually or collectively.

[0080] For example, the electronic device (100) can update the word modification table by obtaining a user word corresponding to at least one edit target word included in the modification text by individually or collectively executing at least one instruction stored in memory (110) by the processor (120).

[0081] According to one embodiment of the present disclosure, the server (2000) may include a memory (210), a processor (220), and a communication interface (230).

[0082] The memory (210) can store a program for processing and controlling the processor (220), and can store data that is input to or output from the server (2000). Additionally, the memory (210) can store data necessary for the operation of the server (2000).

[0083] The memory (210) may include at least one type of storage medium among flash memory type, hard disk type, multimedia card micro type, card type memory (e.g., SD or XD memory, etc.), RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk, and optical disk.

[0084] The memory (210) may not exist separately but may be configured to be included in the processor (220). The memory (210) may be composed of volatile memory, non-volatile memory, or a combination of volatile and non-volatile memory. The memory (210) may store a program or at least one instruction for performing operations according to the embodiments described below. The memory (210) may provide stored data to the processor (220) upon the request of the processor (220).

[0085] The processor (220) controls the overall operation of the server (2000). The processor (220) is configured to control a series of processes to enable the server (2000) to operate according to the embodiments described below, and may be composed of one or more processors. One or more processors included in the processor (220) may be circuitry such as a System on Chip (SoC) or an Integrated Circuit (IC). One or more processors included in the processor (120) may be a general-purpose processor such as a CPU (Central Processing Unit), MPU (Micro Processor Unit), AP (Application Processor), or DSP (Digital Signal Processor); a graphics-dedicated processor such as a GPU (Graphic Processing Unit) or VPU (Vision Processing Unit); an artificial intelligence-dedicated processor such as an NPU (Neural Processing Unit); or a communication-dedicated processor such as a CP (Communication Processor). If one or more processors included in the processor (220) are artificial intelligence dedicated processors, the artificial intelligence dedicated processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0086] The processor (220) can write data to memory (210) or read data stored in memory (210), and in particular, can process data according to a predefined operation rule or artificial intelligence model by executing a program or at least one instruction stored in memory (210). The processor (220) can perform operations described in subsequent embodiments, and operations described as being performed by the server (2000) in subsequent embodiments may be considered to be performed by the processor (220) unless otherwise specified.

[0087] For example, the server (2000) may perform the functions of the server (2000) described in the present disclosure by individually or collectively executing at least one instruction stored in memory (210) by the processor (220). Alternatively, according to one embodiment, the server (2000) may perform the functions described in the present disclosure by individually or collectively executing at least one instruction stored in memory (210) by the processor (220). Accordingly, the processor (220) may perform the operations described in subsequent embodiments, and the operations described as being performed by the server (2000) or detailed components included in the server (2000) in subsequent embodiments may be considered to be performed by the processor (220) unless otherwise specified.

[0088] The communication interface (230) can communicate with at least one electronic device. The communication interface (230) can transmit and receive information for voice recognition and voice assistant services with the electronic device (100) and an external device (not shown). Here, 'communication' may mean the operation of transmitting and / or receiving data, signals, requests, and / or commands, etc. The communication interface (230) can perform wired or wireless communication with at least one electronic device. The server (2000) can communicate with the electronic device (100) through the communication interface (230).

[0089] For example, the communication interface (230) may include at least one of a communication module, a communication circuit, a communication device, an input / output port, and an input / output plug for performing wired or wireless communication with at least one electronic device. For example, the communication interface (230) may include at least one wireless communication module, a wireless communication circuit, or a wireless communication device for performing wireless communication with at least one electronic device.

[0090] For example, the communication interface (230) may include a short-range communication module, such as an IR (infrared) communication module, capable of receiving control commands from a remote controller located at a short distance. In this case, the communication interface (230) may receive control signals from the remote controller.

[0091] For example, the communication interface (230) may include at least one communication module that performs communication according to wireless communication standards such as Bluetooth, Wi-Fi, BLE (Bluetooth Low Energy), NFC (Near Field Communication), RFID (Radio Frequency Identification), Wi-Fi Direct, UWB, or Zigbee. Alternatively, the communication interface (230) may further include a communication module that performs communication with a server to support long-distance communication according to long-distance communication standards. For example, the communication interface (230) may include a communication module that performs communication through a network for internet communication. Additionally, the communication interface (230) may include a communication module that performs communication through a communication network according to communication standards such as 3G, 4G, 5G and / or 6G.

[0092] In one embodiment, the server (2000) can receive voice input from the electronic device (100) by individually or collectively executing at least one instruction stored in the memory (210) by the processor (220). In one embodiment, the server (2000) can perform a function corresponding to automatic voice recognition by individually or collectively executing at least one instruction stored in the memory (210) by the processor (220). In one embodiment, the server (2000) can transmit text corresponding to the voice input obtained from the function corresponding to automatic voice recognition to the electronic device (100) by individually or collectively executing at least one instruction stored in the memory (210) by the processor (220).

[0093] FIG. 3 is a block diagram illustrating a module in which an electronic device according to one embodiment of the present disclosure acquires voice input and outputs text.

[0094] The electronic device (100) may include a security word detection module (310) and a function (320) corresponding to voice recognition. The server (2000) may include a function (2100) corresponding to automatic voice recognition. The function (320) corresponding to voice recognition may include an acoustic model (322) and a language model (324). The function (2100) corresponding to automatic voice recognition may include an acoustic model (2110) and a language model (2120).

[0095] According to one embodiment of the present disclosure, the security word detection module (310) may include appropriate logic, circuits, interfaces, and / or code that can be operated to detect a security word in a voice signal (10).

[0096] According to one embodiment of the present disclosure, the security word detection module (310) and the function (320) corresponding to voice recognition of FIG. 3 may be a software configuration implemented by the processor (120) of the electronic device (100) executing a program stored in memory (110), or it may be a virtual configuration in which no matching hardware device actually exists. Accordingly, the operations described as being performed by the security word detection module (310) and the function (320) corresponding to voice recognition shown in FIG. 3 can actually be seen as being performed by the processor (120) of the electronic device (100) executing a program or instruction stored in memory (110).

[0097] Additionally, at least some of the security word detection module (310) and the voice recognition corresponding function (320) may be implemented to be included in multiple different memories, or a memory containing one module may be implemented to be included in a memory containing another module. Thus, at least some of the security word detection module (310) and the voice recognition corresponding function (320) included in the electronic device according to one embodiment of the present disclosure may be a hardware configuration or a software configuration, and may be implemented in various forms of electronic devices (e.g., a single electronic device or a combination of two or more electronic devices).

[0098] The block diagram of the electronic device (100) illustrated in FIG. 3 is a block diagram for one embodiment. Each component of the block diagram may be integrated, added, or omitted according to the specifications of the actual electronic device being implemented. For example, two or more components may be combined into one component as needed, or one component may be subdivided into two or more components. Furthermore, the functions performed in each block are intended to explain the embodiments, and the specific operations or devices thereof do not limit the scope of the present invention.

[0099] The electronic device (100) can acquire voice input (10). The security word detection module (310) can detect whether the voice input (10) contains a security word. The security word detection module (310) can detect whether the voice input (10) contains a security word within the electronic device (100), which is a local device, without a connection to the server (2000).

[0100] In one embodiment of the present disclosure, a security word detection module (310) can detect whether a security word is included in a voice input (10) by recognizing a pattern of the voice input (10). For example, the security word detection module (310) may include an acoustic model that has learned acoustic information for at least one security word. The security word detection module (310) may measure the similarity (e.g., a confidence score) between the voice input (10) and a voice file corresponding to the security word. For example, the measured similarity may be a similarity determined based on probability information. If the measured similarity is greater than or equal to a threshold similarity, the electronic device (100) may determine that at least one security word is included in the voice input (10). According to one embodiment of the present disclosure, the security word detection module (310) may be learned according to a principle similar to a Wake-up Word Engine (WWE) or a wake-up word recognizer.

[0101] In one embodiment of the present disclosure, the security word detection module (310) can detect whether a security word is included in the voice input (10) by recognizing a word or phrase corresponding to a security word in the voice input (10). For example, the security word detection module (310) may include an ASR engine separate from the function (320) corresponding to voice recognition. The separate ASR engine included in the security word detection module (310) may be a small-scale engine compared to the function (320) corresponding to voice recognition. For example, the security word detection module (310) may include a separate ASR engine based on a Speech-to-Text (STT) model.

[0102] The security word detection module (310) can identify the voice input (10) and determine whether the text corresponding to the voice input (10) contains a security word. For example, the security word detection module (310) can obtain (e.g., measure) the similarity (e.g., confidence score) between the text corresponding to the voice input (10) and the security word. The obtained similarity may be a similarity determined based on probability information. If the similarity obtained by the security word detection module (310) is greater than or equal to a threshold similarity, the electronic device (100) can determine that the voice input (10) contains at least one security word.

[0103] When the security word detection module (310) detects a security word in the voice input (10), the electronic device (100) can perform voice recognition based on a function (320) corresponding to voice recognition.

[0104] The function (320) corresponding to speech recognition may include an acoustic model (322) and a language model (324). The acoustic model (322) may represent a model that models the relationship between speech input and speech units of language. The language model (324) may represent a model that probabilistically predicts words suitable for appearing at specific locations within a sentence by probabilistically calculating how natural the sentence is. The function (320) corresponding to speech recognition may acquire a speech input (10) and output text (330) corresponding to the speech input (10). The electronic device (100) may output text (330).

[0105] The function (320) corresponding to voice recognition can be trained (e.g., updated) without a connection to the server (2000). An embodiment in which the function (320) corresponding to voice recognition is trained (e.g., updated) will be described in more detail with reference to FIGS. 8 to 14.

[0106] If the security word detection module (310) does not detect a security word in the voice input (10), the electronic device (100) can perform voice recognition based on a function (2100) corresponding to automatic voice recognition. The electronic device (100) can transmit the voice input (10) to a server (2000). If the security word detection module (310) does not detect a security word in the voice input (10), the electronic device (100) can transmit the voice input (10) to a server (2000). The function (2100) corresponding to automatic voice recognition can acquire the voice input (10) and acquire text (330) corresponding to the voice input (10). The server (2000) can transmit the text (330) corresponding to the voice input (10) to the electronic device (100). The electronic device (100) can output the text (330).

[0107] Below, we will explain in detail the method for recognizing voice input and obtaining text corresponding to the voice input.

[0108] FIG. 4 is a flowchart of a method in which an electronic device according to one embodiment of the present disclosure acquires voice input and outputs text.

[0109] With reference to FIG. 4, the overall operation of the electronic device (100) of the present disclosure will be described. Additionally, specific details of the operation of the electronic device (100) will be described with reference to the following drawings.

[0110] Referring to FIG. 4, the method of operation of the electronic device (100) may include operations 410 to 440. In one embodiment of the present disclosure, operations 410 to 440 may be executed by at least one processor included in the electronic device (100). The method of operation of the electronic device (100) is not limited to that shown in FIG. 4, and in one or more embodiments, operations not shown in FIG. 4 may be further included, or some operations may be omitted. Since the operations performed by the electronic device (100) have been described above with reference to FIG. 1 to 3, redundant descriptions may be omitted.

[0111] In operation 410, the electronic device (100) can acquire voice input. In one embodiment of the present disclosure, the electronic device (100) can acquire a user's voice input (e.g., speech) through a microphone. The electronic device (100) can acquire a user's voice input (e.g., speech) through a server or another electronic device. The electronic device (100) can acquire voice input by electrically converting the voice input.

[0112] In one embodiment of the present disclosure, the electronic device (100) may preprocess the acquired voice input. For example, if the voice input contains noise, the electronic device (100) may remove the noise within the voice input and obtain a feature vector from the voice input from which the noise has been removed. For example, the electronic device (100) may amplify the voice input to an appropriate level.

[0113] In one embodiment of the present disclosure, an electronic device (100) may generate a feature vector representing the features of a voice input based on a voice input. For example, a device (1000) may extract a feature vector representing the features of a voice input from a voice input. For example, the electronic device (100) may receive data representing the feature vector of a voice input from an external device or server.

[0114] In operation 420, the electronic device (100) can identify whether the voice input contains at least one security word. The electronic device (100) can detect at least one security word contained in the voice input.

[0115] In one embodiment of the present disclosure, the electronic device (100) can identify whether the voice input contains at least one security word based on a predefined security word recognition model. In one embodiment of the present disclosure, the electronic device (100) can identify whether the voice input contains at least one security word based on a security word recognition model stored in the electronic device (100).

[0116] Depending on whether the voice input contains at least one security word, the electronic device (100) may choose whether to perform voice recognition based on a function corresponding to voice recognition included in the electronic device for the voice input, or to perform voice recognition based on a function corresponding to automatic voice recognition included in the server (e.g., a cloud automatic voice recognition engine).

[0117] For example, the electronic device (100) may determine whether to transmit the voice input to the server (2000) depending on whether the voice input contains at least one security word. If the voice input contains at least one security word, the electronic device (100) may decide not to transmit the voice input to the server (2000). If the voice input does not contain at least one security word, the electronic device (100) may decide to transmit the voice input to the server (2000). The detailed operation of operation 420 will be described in detail with reference to FIG. 5, and redundant descriptions are omitted here for brevity.

[0118] In operation 430, the electronic device (100) can perform a function corresponding to speech recognition for the voice input if the acquired voice input contains at least one security word. For example, the electronic device (100) can switch (or change) the speech recognition mode to an on-device mode if the voice input contains at least one security word. In the on-device mode, the electronic device (100) can perform speech recognition based on a function corresponding to speech recognition (e.g., on-device automatic speech recognition, on-device automatic speech recognition, on-device ASR engine).

[0119] For example, when the electronic device (100) detects voice input containing a security word, it can enhance privacy protection by automatically switching to an on-device mode where the voice input can be processed on the device itself without being transmitted to an external server. For example, in the switched on-device mode, the electronic device (100) processes the user's voice input in a local environment and minimizes the possibility of voice data being leaked externally.

[0120] In one embodiment of the present disclosure, the electronic device (100) can output text corresponding to a voice input obtained based on a function corresponding to voice recognition. In one embodiment of the present disclosure, the electronic device (100) can output text corresponding to a voice input through a display of the electronic device (100). In one embodiment of the present disclosure, the electronic device (100) can output text corresponding to a voice input through a display of an external device connected to the electronic device (100).

[0121] In operation 440, the electronic device (100) can transmit voice input to the server (2000). In one embodiment of the present disclosure, if the voice input does not contain at least one security word, the electronic device (100) can control the voice input to perform voice recognition based on a function corresponding to automatic voice recognition included in the server (e.g., a Cloud Automatic Speech Recognition (Cloud ASR) engine). For example, if the voice input contains at least one security word, the electronic device (100) can switch (or change) the voice recognition mode to a cloud mode. In the cloud mode, the electronic device (100) can perform voice recognition based on the cloud automatic speech recognition engine.

[0122] For example, if a security word is not detected, the electronic device (100) can be controlled to switch to cloud mode to perform more sophisticated voice analysis and complex command processing (e.g., conversational question-and-answer, multi-language translation) on the server (2000). By switching to cloud mode, the electronic device (100) can provide high-accuracy voice recognition results to compensate for the resource limitations of the electronic device (100).

[0123] For example, an electronic device (100) can transmit voice input to a cloud automatic voice recognition engine. The electronic device (100) can transmit voice input to a server (2000) or a cloud automatic voice recognition engine included in the server (2000) via a communication network conforming to a communication standard such as wired or wireless communication, for example, Bluetooth, Wi-Fi, BLE (Bluetooth Low Energy), NFC (Near Field Communication), RFID (Radio Frequency Identification), Wi-Fi Direct, UWB, or Zigbee, the Internet, 3G, 4G, 5G and / or 6G.

[0124] In one embodiment, the electronic device (100) may be controlled to output text corresponding to a voice input based on a function corresponding to automatic voice recognition performed on a server for voice input. For example, the electronic device (100) may receive text corresponding to a voice input from a server (2000). The electronic device (100) may receive text corresponding to a voice input from the server (2000) or a function corresponding to automatic voice recognition included in the server (2000). The electronic device (100) may receive text corresponding to a voice input obtained from a function (2100) corresponding to automatic voice recognition of the server (2000) from the server (2000). The electronic device (100) may receive text corresponding to a voice input from another external electronic device connected to the server (2000).

[0125] In one embodiment of the present disclosure, the electronic device (100) can output text corresponding to voice input through a display of the electronic device (100). In one embodiment of the present disclosure, the electronic device (100) can output text corresponding to voice input through a display of an external device connected to the electronic device (100).

[0126] FIG. 5 is a flowchart illustrating a detailed operation for identifying whether at least one security word is included in a voice input included in operation 420 according to one embodiment of the present disclosure.

[0127] In operation 422, the electronic device (100) can measure (or obtain, calculate, determine) the similarity between voice input and at least one security word.

[0128] According to one embodiment of the present disclosure, an electronic device (100) can calculate (or, calculate, obtain) a confidence score of an input voice input and a security word. The confidence score is a value between 0 and 1 and may be a probability value indicating how similar the input signal is to the security word. For example, the confidence score may include at least one of acoustic similarity, which is the similarity between the pronunciation of the security word and the voice input, or contextual similarity, which is the semantic relationship between the security word and the words included in the voice input. The electronic device (100) may compensate for the effect of ambient noise on the recognition of the security word when obtaining the confidence score.

[0129] According to one embodiment of the present disclosure, an electronic device (100) can measure (or obtain, calculate, determine) the similarity between a voice input and at least one security word based on a security word recognition function. The electronic device (100) may include a security word recognition function. In one embodiment of the present disclosure, the electronic device (100) can detect whether a security word is included in the voice input based on a security word recognition model without a connection to a server (2000) or an external device. The security word recognition function may be a function separate from the function corresponding to voice recognition.

[0130] For example, the electronic device (100) can obtain a similarity between a user's speech or voice input and at least one security word based on an acoustic model that has learned acoustic information for at least one security word. The electronic device (100) can identify whether the received voice input contains at least one security word based on probability information corresponding to the determined similarity. For example, the electronic device (100) can obtain a similarity with the voice input corresponding to the security word by recognizing the pattern of the voice input (e.g., tone, pitch, intonation, or speed). The electronic device (100) can identify whether the received voice input contains at least one security word based on the calculated similarity. In this case, the electronic device (100) may include a pattern recognizer or a call word recognizer for detecting the pattern of the voice input corresponding to the security word.

[0131] For example, the electronic device (100) can obtain (or determine) the similarity between a user's speech or voice input and at least one security word by detecting whether the text corresponding to the acquired voice input contains a security word. For example, the electronic device (100) can identify whether the received voice input contains at least one security word that is pre-set, based on the similarity between the security word text and the text corresponding to the voice input. In this case, the electronic device (100) may include a low-capacity on-device automatic speech recognition model for detecting security words. For example, the electronic device (100) may include a separate low-capacity STT (Speech to Text) engine or a function corresponding to speech recognition for detecting security words.

[0132] In operation 424, the electronic device (100) can identify (or determine, judge) whether the similarity between the voice input and at least one security word is greater than or equal to a threshold similarity.

[0133] A threshold similarity according to one embodiment of the present disclosure may be a preset value. The threshold similarity may be adaptively adjusted by a processor (120). The threshold similarity may be adaptively adjusted through a communication interface (130) or an input / output interface, but is not limited thereto. For example, an electronic device (100) may identify that the similarity between a voice input and at least one security word is greater than or equal to the threshold similarity when the similarity between the voice input and at least one security word is 0.93 and the threshold similarity is 0.9.

[0134] In operation 426, the electronic device (100) can identify (or determine) that the voice input contains at least one security word if the similarity between the voice input and at least one security word is greater than or equal to a threshold similarity. After operation 426, the electronic device (100) can perform operation 430, which performs voice recognition based on an on-device automatic voice recognition model.

[0135] In operation 428, the electronic device (100) can identify (or determine) that the voice input does not contain at least one security word if the similarity between the voice input and at least one security word is less than a threshold similarity. After operation 428, the electronic device (100) can perform operation 440, which performs voice recognition based on a function corresponding to voice recognition.

[0136] FIG. 6 is a flowchart for describing a detailed operation for identifying whether at least one security word is included in the voice input included in operation 420 when a user input that modifies a security word is obtained according to one embodiment of the present disclosure.

[0137] In operation 610, the electronic device (100) can obtain user input corresponding to a modification of at least one security word. By obtaining user input corresponding to a modification of at least one security word, the electronic device (100) can update (e.g., update) at least one security word.

[0138] In one embodiment of the present disclosure, the electronic device (100) may obtain user input corresponding to editing the security word itself. For example, the electronic device (100) may obtain user input that edits the existing security word 'secure green' to 'secure red'. In one embodiment of the present disclosure, the electronic device (100) may obtain user input corresponding to editing at least one security word itself. The electronic device (100) may obtain user input corresponding to editing each security word. The electronic device (100) may obtain user input that individually edits at least one security word.

[0139] In one embodiment of the present disclosure, the electronic device (100) may obtain user input corresponding to the addition of a security word. For example, it may obtain user input adding the security word 'person assistant' to the existing security words 'secure green', 'star 9', and 'smart intelligence'. In one embodiment of the present disclosure, the electronic device (100) may obtain user input corresponding to the addition of at least one security word. The electronic device (100) may obtain user input corresponding to the editing of a security word list. The electronic device (100) may obtain user input adding a security word to a security word list. For example, the electronic device (100) may obtain user input adding at least one security word to an existing set of security words.

[0140] In one embodiment of the present disclosure, the electronic device (100) may obtain user input corresponding to the deletion of a security word. For example, from existing security words 'secure green', 'star 9', 'smart intelligence', the electronic device (100) may obtain user input for deleting the security word 'star 9'. In one embodiment of the present disclosure, the electronic device (100) may obtain user input corresponding to the deletion of at least one security word. User input corresponding to the deletion of a security word may be replaced with user input for editing a security word list, or user input for deleting a security word from a security word list. For example, the electronic device (100) may obtain user input for deleting at least one security word from an existing set of security words.

[0141] For example, the electronic device (100) may obtain user input for editing, adding, or deleting at least one security word through the input / output interface of the electronic device (100). For example, the electronic device (100) may obtain user input for editing, adding, or deleting at least one security word through a keyboard, touch screen, microphone, etc., but is not limited to the examples described.

[0142] In operation 620, the electronic device (100) can identify whether at least one security word reflecting user input is included in the voice input. For example, when the electronic device (100) identifies whether one security word is included in the voice input, it can reflect modifications and edits to the security word through user input. When the electronic device (100) identifies the security word, modifications and edits to the security word through user input can be automatically (or in real time) reflected in the security word.

[0143] In one embodiment of the present disclosure, at least one security word reflecting user input may be a security word modified according to the operation described above with reference to operation 610. For example, at least one security word reflecting user input may be a security word in which an existing security word has been edited, added, or deleted.

[0144] The electronic device (100) can perform operation 430 based on the determination that at least one security word reflecting user input is included in the voice input. The electronic device (100) can perform operation 440 based on the determination that at least one security word reflecting user input is not included in the voice input.

[0145] In one embodiment of the present disclosure, the operation of an electronic device (100) identifying whether at least one security word reflecting user input is included in voice input may correspond to the operation 420 described above with reference to FIGS. 4 and FIGS. 5. That is, the electronic device (100) may identify at least one security word (or, a list of security words) updated by user input and identify whether at least one security word is included in voice input.

[0146] In one embodiment of the present disclosure, the electronic device (100) may perform operation 610 before operation 410 or after operation 410. For example, the electronic device (100) may, after acquiring voice input (operation 410), acquire user input corresponding to the modification of at least one security word (operation 610), and identify whether at least one security word reflecting the user input is included in the voice input (operation 620). For example, the electronic device (100) may acquire user input corresponding to the modification of at least one security word (operation 610), and after acquiring voice input (operation 410), identify whether at least one security word reflecting the user input is included in the voice input (operation 620).

[0147] FIG. 7 is a reference diagram for showing types of security words according to one embodiment of the present disclosure.

[0148] Example 1, 710, shows an example where the security word is the trigger word.

[0149] According to one embodiment of the present disclosure, a trigger word may refer to a word that serves as a criterion for determining the initiation of speech recognition. The trigger word may include at least one pre-set trigger word. The trigger word may be a call word or a speech recognition start command. In this specification, a call word or a speech recognition start command may also be referred to as a trigger word.

[0150] According to one embodiment of the present disclosure, a security word may be included in the trigger word of the electronic device (100). For example, the security word (712) included in the trigger word may represent a word that serves as a criterion for determining the initiation of voice. For example, when the electronic device (100) recognizes the security word (712) included in the trigger word in the voice input (710), the electronic device (100) may perform voice recognition through a function corresponding to voice recognition.

[0151] In one embodiment of the present disclosure, the electronic device (100) may simultaneously initiate voice recognition and activate a function corresponding to voice input based on a function corresponding to voice recognition, and may perform voice recognition for a voice input based on a function corresponding to voice recognition after initiating (or activating) voice recognition.

[0152] For example, the electronic device (100) can initiate voice recognition and simultaneously activate a function corresponding to voice recognition for voice input based on an on-device automatic voice recognition model.

[0153] In one embodiment of the present disclosure, when the electronic device (100) identifies a security word (712) included in the trigger word in the voice input (710), the electronic device (100) can perform a function corresponding to voice recognition from the time voice recognition is initiated. The electronic device (100) can perform a function corresponding to voice recognition based on an on-device automatic voice recognition model for the entire voice input (710).

[0154] In one embodiment of the present disclosure, when the electronic device (100) recognizes a trigger word that is not a security word in a voice input, the electronic device (100) may initiate voice recognition based on any automatic voice recognition model. For example, the electronic device (100) may initiate voice recognition based on a function corresponding to voice recognition, a function corresponding to automatic voice recognition included in a server for voice recognition, or a hybrid automatic voice recognition function of the electronic device (100) and the server.

[0155] For example, the trigger word of the electronic device (100) may include "Alpino, Ecobricks, Secure Green." Among the trigger words, the security word "Secure Green" may be included. When the electronic device (100) recognizes the trigger word Alpino, Ecobricks, or Secure Green in the voice input (710), it may perform (or initiate) a function corresponding to the voice recognition. When the electronic device (100) recognizes the security word (712) Secure Green included in the trigger word in the voice input (710), it may perform (or initiate) a function corresponding to the voice recognition.

[0156] For example, the electronic device (100) can enable a function corresponding to voice recognition.

[0157] In one embodiment of the present disclosure, the electronic device (100) can activate a function corresponding to voice recognition for the voice input based on a trigger word included in the voice input.

[0158] In one embodiment of the present disclosure, an electronic device (100) can identify whether a trigger word that has activated voice recognition for a voice input is a security word. If the trigger word is included in at least one security word, the electronic device (100) can output text corresponding to the voice input by performing voice recognition on the entire voice input (710) using a function corresponding to voice recognition included in the electronic device (100). For example, the electronic device (100) can output text corresponding to the entire voice input (710) using a function corresponding to voice recognition. If the trigger word is included in at least one security word, the electronic device (100) can enhance the security of the voice input requiring security by not transmitting the entire voice input (710) to a server (2000).

[0159] Example 2, 720, shows an example where the security word is any word other than the trigger word.

[0160] The electronic device (100) can identify whether at least one security word is included in a text stream (720) accumulated in units of a predetermined interval of voice input. For example, if at least one security word is included at least once in a text stream (720) accumulated in units of a predetermined interval of voice input, the electronic device (100) can perform voice recognition using a function corresponding to voice recognition. For example, if at least one security word is identified in the text stream (720), the electronic device (100) can control the function corresponding to automatic voice recognition included in an existing server to stop and control the function corresponding to voice recognition included in the electronic device (100) to perform. In one embodiment, the electronic device (100) can control the output of text corresponding to the voice input.

[0161] In one embodiment of the present disclosure, the security word may include proper nouns such as a specific company name, the name of a specific part, the name of a specific project, the name of a specific product, etc. In one embodiment, the security word may represent a word that is generally included in situations where security is required as a result of learning. In one embodiment, the security word may include words generally associated with personal information, such as a resident registration number, an address, or a mobile phone number. For example, the security word may include the part name 'C1234' (726).

[0162] For example, the security word may include a word obtained by an artificial intelligence model (e.g., a generative artificial intelligence model). For example, the security word may include 'newly introduced part' (724). 'Newly introduced part' is not a proper noun, but may be a word generally included in a meeting situation regarding a specific part where security is required. In this case, the electronic device (100) may include 'newly introduced part' (724) in the security word through the learning of the artificial intelligence model.

[0163] When at least one security word is included in a text stream accumulated in units of a predetermined interval of voice input, the electronic device (100) can perform voice recognition using a function corresponding to voice recognition included in the electronic device for the interval and the voice input corresponding thereafter. In one embodiment, the electronic device (100) can be controlled to output text corresponding to the voice input.

[0164] For example, the electronic device (100) can identify whether at least one security word is included in a text stream accumulated in increments of about 5 seconds. The electronic device (100) can determine that at least one security word is included in a text stream of a specific interval (e.g., between 1 minute 30 seconds and 1 minute 35 seconds of voice input). The electronic device (100) can perform voice recognition using a function corresponding to voice recognition for voice inputs corresponding to the interval from 1 minute 30 seconds to the last interval. By determining whether a security word is included by dividing it into intervals, the electronic device (100) can effectively identify a security word included in the middle of the voice input while reducing memory usage and computational requirements.

[0165] The operation of the electronic device (100) described with reference to the first example 710 and the second example 720 above may be included in the process of determining whether the voice input contains at least one security word, as described above with reference to operation 420 of FIG. 4 and FIG. 5. For example, whether the security word is a trigger word or not, the electronic device (100) may determine whether the voice input contains at least one security word by obtaining the similarity between the security word and the voice input.

[0166] FIG. 8 is a schematic diagram illustrating a method for outputting text corresponding to a voice input based on a function corresponding to voice recognition according to one embodiment of the present disclosure.

[0167] The electronic device (100) can acquire voice input (10). The electronic device (100) can acquire voice input (10) based on the speech of the user (1). The electronic device (100) can perform a voice input recognition operation in the electronic device (100) as the voice input (10) contains a security word.

[0168] The electronic device (100) can perform voice recognition for a voice input (10) based on a function (320) corresponding to voice recognition. The electronic device (100) can acquire (or output) text (332) corresponding to the voice input (10) based on a function (320) corresponding to voice recognition.

[0169] The electronic device (100) can modify text (332) based on a word modification table (810). The word modification table (810) may include a word to be modified, which is a word that needs to be modified, and a replacement word, which is a word to which the word to be modified has been corrected. The electronic device (100) can change the word to be modified included in text (332) to a replacement word based on the word modification table (810). The electronic device (100) can obtain a modified text (812) in which the word to be modified included in text (332) has been changed to a replacement word.

[0170] The function (320) corresponding to voice recognition may include a word correction table (810). That is, within the function (320) corresponding to voice recognition, text (332) is primarily obtained, and a corrected text (812) based on the text (332) can be obtained based on the word correction table (810) included in the function (320) corresponding to voice recognition.

[0171] For example, the electronic device (100) can change a word to be modified (e.g., "Bisionsection") included in text (332) to a replacement word (e.g., "Visionsection") based on a word modification table (810). The electronic device (100) can identify "Bisionsection" included in text (332) from the words to be modified in the word modification table (810). The electronic device (100) can identify "Visionsection," which is a replacement word corresponding to "Bisionsection," in the word modification table (810). The electronic device (100) can obtain a modified text (812) in which the word to be modified (e.g., "Bisionsection") in text (332) is changed to a replacement word (e.g., "Visionsection").

[0172] The electronic device (100) can train a function (320) corresponding to speech recognition by changing the word to be modified into a replacement word a threshold number of times based on a word modification table (810). For example, the electronic device (100) can train the probability of recognizing a replacement word in any speech input based on the characteristics of the speech input corresponding to the word to be modified. For example, the electronic device (9100) can train the probability of recognizing a replacement word in any text based on the combination order of at least one word including a replacement word included in the modified text (812).

[0173] In one embodiment of the present disclosure, an electronic device (100) may obtain a user word (820). The electronic device (100) may obtain a result text (822) based on the user word (820). The electronic device (100) may obtain (or output) the result text (822) based on input from a user (1) who changes the word to be edited included in the modified text (812) to the user word (820).

[0174] For example, the electronic device (100) can change the word to be edited (e.g., "Lexyfeat") included in the modified text (812) to a user word (e.g., "Lexifeat"). The electronic device (100) can identify at least one word included in the modified text (812). The electronic device (100) can obtain user input identifying the word to be edited among at least one word included in the modified text (812). The electronic device (100) can obtain a user word (820) corresponding to the word to be edited among at least one word included in the modified text (812). The electronic device (100) can obtain a result text (822) in which the word to be edited (e.g., "Lexyfeat") is changed to a user word (e.g., "Lexifeat") in the text (332).

[0175] The electronic device (100) can update the word modification table (810) as it receives user input to change the word to be edited to a user word (820). For example, the electronic device (100) can add the word to be edited to the word to be modified in the word modification table (810). For example, the electronic device (100) can add the user word to the replacement word in the word modification table (810).

[0176] FIG. 9 is a reference diagram for illustrating a word correction table according to one embodiment of the present disclosure.

[0177] According to one embodiment of the present disclosure, an electronic device (100) may include a word correction table (900). The word correction table may include at least one word to be corrected and at least one replacement word. The word correction table may include a pair of a word to be corrected and a replacement word corresponding to the word to be corrected. For example, the word correction table may include a history of correction (or, correction) of the word to be corrected. For example, the word correction table may include a history (or, result) of conversion of the word to be corrected to a replacement word.

[0178] The replacement words may include words that are likely to be misrecognized by the electronic device (100). For example, the replacement words may include words that are likely to be misrecognized by the voice recognition function included in the electronic device (100). For example, the replacement words may include words not in the dictionary (Out of Vocabulary, OoV), proper nouns such as names, company names, content names, technical terms, or abbreviations.

[0179] The word to be corrected may include a word in which the replacement word is misidentified. For example, the word to be corrected may include a word in which the replacement word is misidentified by a function corresponding to speech recognition included in the electronic device (100). For example, the word to be corrected may include a word in which the replacement word is misidentified by an automatic speech recognition engine included in any other electronic device or server.

[0180] For example, the word correction table (900) may include 'Metabos' as a word to be corrected. The word correction table (900) may include 'Metaverse' as a replacement word. The word to be corrected, 'Metabos', may be a word in which the replacement word, 'Metaverse', is misrecognized. The word to be corrected, 'Metabos', may be a word in which the replacement word, 'Metaverse', is misrecognized by the electronic device (100) or any other electronic device.

[0181] The alternative term 'metaverse' refers to a three-dimensional virtual world or a realistic four-dimensional virtual space-time connected to real life and legally recognized activities such as jobs, finance, and learning. It is a neologism formed by combining 'meta,' meaning virtual or transcendent, and 'universe,' meaning world or universe. As 'metaverse' is a neologism, it may be included in the Out of Vocabulary (OoV) category. Since 'metaverse' is a technical term or proper noun representing a virtual space-time, it may be included in the Out of Vocabulary (OoV) category. When an electronic device (100) acquires a voice input containing 'metaverse,' the accuracy of the voice recognition result for 'metaverse' may decrease. For example, an electronic device (100) may recognize 'metaverse' included in the Out of Vocabulary (OoV) words as 'metaboss', 'metaberse', 'mettbeso', or 'metabirth'. The words to be modified may include 'metaboss', 'metaberse', 'mettbeso', or 'metabirth'. The replacement word corresponding to the words to be modified may be 'metaverse'.

[0182] For example, the word correction table (900) may include 'Samsung' as a word to be corrected. The word correction table (900) may include 'Samsung' as a replacement word. The word to be corrected, 'Samsung', may be a word in which the replacement word, 'Samsung', is misrecognized. The word to be corrected, 'Samsung', may be a word in which the replacement word, 'Samsung', is misrecognized by the electronic device (100) or any other electronic device. Since 'Samsung' is a proper noun representing a company name, it may be an Out of Vocabulary (OoV) word.

[0183] In one embodiment of the present disclosure, the word correction table (900) may contain one replacement word corresponding to a plurality of correction target words. For example, there may be a replacement word 'Lexifeat' corresponding to the correction target words 'Lexyfeat' and 'Legen'. Since the electronic device (100) may misrecognize replacement words in various ways, there may be a plurality of correction target words corresponding to one replacement word.

[0184] In one embodiment of the present disclosure, the electronic device (100) may include a predetermined word correction table. For example, the word correction table may be obtained by the electronic device (100) from a server. The word correction table may be a value obtained based on a word misrecognized by any other electronic device and a word in which the misrecognized word has been corrected. By including the predetermined word correction table, the electronic device (100) can identify word correction history based on user input from other electronic devices or other users. The electronic device (100) can improve the accuracy of speech recognition by modifying the speech recognition results of a function corresponding to speech recognition based on the predetermined word correction table. The electronic device (100) can improve the accuracy of speech recognition for replacement words (e.g., proper nouns) with less user input by modifying the speech recognition results of a function corresponding to speech recognition based on the predetermined word correction table.

[0185] In one embodiment of the present disclosure, an electronic device (100) can modify text based on a function corresponding to speech recognition based on a word modification table (900). For example, the electronic device (100) can change the word to be modified included in the text to a replacement word based on the word modification table. For example, the electronic device (100) can obtain a modified text in which the word to be modified included in the text is changed to a replacement word based on the word modification table. Since the above operation has been described in detail with reference to operation 1010 of FIG. 10, redundant descriptions are omitted here for brevity.

[0186] In one embodiment of the present disclosure, the electronic device (100) may update (or, update) the word modification table (900). The electronic device (100) may update (or, update) the word modification table based on the operation described with reference to FIGS. 12 to 14 below.

[0187] For example, the electronic device (100) can obtain a user word corresponding to at least one word to be edited included in the edit text. The electronic device (100) can update (or update) a word modification table based on the word to be edited and the user word corresponding to the word to be edited. For example, the electronic device (100) can add the word to be edited and the user word corresponding to the word to be edited to the word modification table.

[0188] For example, the electronic device (100) can obtain user input that changes the word to be edited, 'memory', included in the edit text, to the user word 'memory'. The electronic device (100) can add the word to be edited, 'memory', to the word to be edited in the word correction table. The electronic device (100) can add the user word 'memory', which corresponds to the word to be edited, to the replacement words in the word correction table.

[0189] The electronic device (100) can further improve the accuracy of voice recognition and perform user-customized voice recognition functions by updating the word correction table. By updating the word correction table, the electronic device (100) can quickly and easily improve the accuracy of voice recognition even before updating the function corresponding to voice recognition. The electronic device (100) can prevent the leakage of security words and simultaneously improve the accuracy of voice recognition for security words by updating (or updating) the word correction table within the local device without a connection to a server.

[0190] FIG. 10 is a flowchart illustrating a method for training a function corresponding to speech recognition according to one embodiment of the present disclosure.

[0191] In operation 1010, the electronic device (100) can identify (or determine, decide) whether a word to be modified included in the text has been changed to a replacement word more than a threshold number of times based on a word modification table included in the electronic device. In one embodiment, the word to be modified may represent a word that needs to be modified. In one embodiment, the replacement word may represent a word that the electronic device (100) will use after modification in relation to the word to be modified.

[0192] In one embodiment, the threshold number may be a number specified by the system (e.g., 3 times). In one embodiment, the threshold number may be a number specified by the user (e.g., 5 times). In one embodiment, the threshold number may be a number that varies adaptively.

[0193] The electronic device (100) can identify (or determine) whether a word to be modified included in the text has been changed to a replacement word more than a threshold number of times based on a word modification table included in the electronic device.

[0194] In operation 1020, the electronic device (100) can train (or update) a function corresponding to voice recognition. In one embodiment of the present disclosure, the electronic device (100) can train (or update) a function corresponding to voice recognition when modifications to a specific word occur more than a threshold number of times. The electronic device (100) can train (or update) a function corresponding to voice recognition within a local device without a connection to a server.

[0195] In one embodiment of the present disclosure, an electronic device (100) can train (or update) a function corresponding to speech recognition when a modification to a specific word occurs one or more times. Through this, the electronic device (100) can train (or update) a function corresponding to speech recognition when the same modification to a specific word is performed multiple times. Through this, the electronic device (100) can increase the accuracy of the function corresponding to speech recognition and perform updates carefully. For example, the electronic device (100) may not train (or update) a function corresponding to speech recognition when a modification to a specific word is performed only once.

[0196] For example, the electronic device (100) can train at least one of an acoustic model or a language model included in a function corresponding to speech recognition. For example, the electronic device (100) can train a function corresponding to speech recognition that is an end-to-end speech recognition model having a structure that includes an integrated neural network without separately including an acoustic model, a pronunciation dictionary, and a language model.

[0197] For example, the electronic device (100) can update an acoustic model based on a stored voice file or feature points extracted from a voice file. For example, the electronic device (100) can update a language model based on a word combination order. The process of the electronic device (100) training at least one of the acoustic model or the language model will be described in more detail with reference to FIGS. 11a through 11c.

[0198] FIG. 11a is a flowchart illustrating an operation for training a function corresponding to voice recognition included in operation 1020 according to one embodiment of the present disclosure.

[0199] In operation 1110, the electronic device (100) can train an acoustic model included in a function corresponding to speech recognition. The electronic device (100) may include a pre-trained acoustic model. The electronic device (100) may further train the pre-trained acoustic model. For example, the electronic device (100) may update the probability of obtaining a replacement word from a speech input. For example, the electronic device (100) may update the probability of obtaining a replacement word from a speech input based on the characteristics of at least one speech input containing a word to be modified.

[0200] In one embodiment of the present disclosure, the electronic device (100) can change a word to be modified included in text to a replacement word based on a word modification table. In one embodiment of the present disclosure, the electronic device (100) can obtain a voice input corresponding to the word to be modified by changing a word to be modified included in text to a replacement word based on a word modification table.

[0201] In one embodiment of the present disclosure, the electronic device (100) can identify characteristics of a voice input corresponding to a word to be modified. For example, the electronic device (100) can identify the frequency, frequency spectrum, inter-phoneme spacing, intonation of pronunciation, etc. of the voice input corresponding to the word to be modified.

[0202] For example, the electronic device (100) can acquire voice inputs corresponding to modified text acquired more than a threshold number of times. Since the electronic device (100) updates the device automatic speech recognition model as text is changed more than a threshold number of times based on a word modification table, the electronic device (100) can acquire voice inputs corresponding to modified text more than a threshold number. Accordingly, the 'voice input' included in the description of FIGS. 11a to 11c can be replaced with 'at least one voice input' or 'more than a threshold number of voice inputs'.

[0203] The electronic device (100) can train the probability of recognizing a replacement word from any voice input. In one embodiment of the present disclosure, the electronic device (100) can update the probability of recognizing a replacement word in the voice input based on the pattern of voice input corresponding to the modified text and the correlation between the word to be modified and the replacement word. In one embodiment of the present disclosure, the electronic device (100) can update the probability of recognizing a replacement word in the voice input based on the pattern of voice input corresponding to the replacement word and the correlation between the word to be modified and the replacement word.

[0204] In one embodiment of the present disclosure, the electronic device (100) may add the recognition probability of an alternative word to an existing acoustic model. In one embodiment of the present disclosure, the electronic device (100) may train, adapt, or update the existing acoustic model based on the recognition probability of an alternative word. The electronic device (100) may train the acoustic model based on voice input for a word that is not in the existing acoustic model. The electronic device (100) may train the acoustic model based on voice input for a word that the existing acoustic model does not recognize well.

[0205] For example, an electronic device (100) may acquire at least one modified text and a voice input corresponding to the modified text, where the word to be modified is 'BTcoin' and the replacement word is 'Bitcoin'. The electronic device (100) may learn the characteristics or patterns of the voice input recognized as "BTcoin". Based on the characteristics or patterns of the voice input recognized as "BTcoin", the electronic device (100) may update the probability that the voice input will be recognized as "Bitcoin". The electronic device (100) may train an acoustic model included in an on-device automatic speech recognition model within the electronic device (100) without a connection to a server.

[0206] In operation 1120, the electronic device (100) can train a language model included in a function corresponding to speech recognition. The electronic device (100) may include a pre-trained language model. The electronic device (100) may further train the pre-trained language model. For example, the electronic device (100) may update the probability of obtaining a replacement word from any text.

[0207] In one embodiment of the present disclosure, an electronic device (100) can change a word to be modified included in text into a replacement word based on a word modification table. In one embodiment of the present disclosure, the electronic device (100) can obtain a modified text by changing a word to be modified included in text into a replacement word based on a word modification table. In one embodiment of the present disclosure, the electronic device (100) can identify a replacement word included in the modified text. For example, the electronic device (100) can identify the position of the replacement word included in the modified text, the word composition before the replacement word, or the word composition after the replacement word. For example, the electronic device (100) can identify the combination order of at least one word including the replacement word included in the modified text.

[0208] For example, the electronic device (100) can identify a combination order of at least one word including a replacement word included in the modified text obtained more than a threshold number of times. Since the electronic device (100) updates the device automatic speech recognition model as the text is changed more than a threshold number of times based on the word modification table, the electronic device (100) can obtain more than a threshold number of modified texts. Accordingly, the 'modified text' included in the description of FIGS. 11a through 11c can be replaced with 'at least one modified text' or 'more than a threshold number of modified texts'.

[0209] In one embodiment of the present disclosure, the electronic device (100) may update the probability of obtaining a replacement word from any text based on the combination order of at least one word included in the modified text. For example, the electronic device (100) may update the probability of obtaining a replacement word from any text based on at least one of the replacement word and the word preceding the replacement word or the word following the replacement word.

[0210] For example, "word preceding the replacement word" may include the word immediately preceding the replacement word that is adjacent to the replacement word. "Word preceding the replacement word" may include not only the word immediately preceding the replacement word that is adjacent to the replacement word, but also at least one word located earlier than the "replacement word" in the modified text.

[0211] For example, "word following the replacement word" includes the word immediately following the replacement word that is adjacent to the replacement word. "Word following the replacement word" may include not only the word immediately following the replacement word that is adjacent to the replacement word, but also at least one word located after the "replacement word" in the modified text.

[0212] In one embodiment of the present disclosure, the electronic device (100) may add probabilities for replacement words to an existing language model. In one embodiment of the present disclosure, the electronic device (100) may train, adapt, or update the existing language model based on probabilities for replacement words. In one embodiment of the present disclosure, the electronic device (100) may analyze (e.g., identify, calculate) at least one of the frequency or order of word combinations included in the modified text. The electronic device (100) may learn linguistic patterns included in the modified text.

[0213] For example, an electronic device (100) may obtain at least one modified text in which the word to be modified is 'Bitcoin' and the replacement word is 'Bitcoin'. For example, the electronic device (100) may obtain modified text such as "I bought Bitcoin," "Bitcoin is breaking record highs day by day," or "Cryptocurrencies such as Bitcoin are not fiat currency." The electronic device (100) may update the probability of obtaining the replacement word "Bitcoin" in any text based on the combination order of words such as "bought," "highest price," "cryptocurrency," and "fiat currency" included in the modified text. For example, the electronic device (100) may update the probability of "Bitcoin" being replaced with "Bitcoin" in any text such as "As many people buy Bitcoin, there is a growing voice that Bitcoin should be designated as fiat currency." The electronic device (100) can update the probability that "Bitcoin" appears at the location of "Bitcoin" within the sentence. The electronic device (100) can train a language model included in a function corresponding to speech recognition within the electronic device (100) without a connection to a server.

[0214] The electronic device (100) can train at least one of an acoustic model or a language model included in a function corresponding to speech recognition within the electronic device (100) without a connection to a server. The electronic device (100) can improve the accuracy of an on-device automatic speech recognition model.

[0215] The electronic device (100) can train a function that responds to voice recognition in a user-customized manner. For example, the electronic device (100) can train a function that responds to voice recognition in a user-customized manner based on the pronunciation, intonation, and speaking habits that the user frequently uses. For example, the electronic device (100) can train a function that responds to voice recognition based on words that the user frequently modifies or frequently uses. For example, the electronic device (100) can enhance security by training at least one of an acoustic model or a language model included in a function that responds to voice recognition based on security words that the user frequently inputs, while not transmitting the security words to a server. The electronic device (100) can prioritize recommending words or phrases that are frequently used in a specific user environment.

[0216] FIG. 11b is a reference diagram for illustrating an operation of training an acoustic model among the functions corresponding to speech recognition included in operation 1020 according to one embodiment of the present disclosure.

[0217] A first embodiment 1130 represents a voice input including the alternative word Lexifeat. An electronic device (100) may acquire at least one voice input corresponding to the alternative word. For example, the electronic device (100) may acquire a first voice input (1132) corresponding to the alternative word Lexifeat and a second voice input (1134) corresponding to the alternative word Lexifeat.

[0218] The first voice input (1132) and the second voice input (1134) may be the same signal or different signals. Even if the first voice input (1132) and the second voice input (1134) are voice inputs for the same word "Lexifeat," they may have different characteristics depending on the speaker's pronunciation, intonation, speaking speed, age, gender, etc.

[0219] The electronic device (100) can extract features of the first voice input (1132). The electronic device (100) can extract features of the second voice input (1134). The electronic device (100) can train an acoustic model based on the features of the first voice input (1132) and the features of the second voice input (1134). The electronic device (100) can train the probability of recognizing the alternative word Lexifeat from any voice input.

[0220] FIG. 11c is a reference diagram for illustrating an operation of training a language model among the functions corresponding to speech recognition included in operation 1020 according to one embodiment of the present disclosure.

[0221] The electronic device (100) can train a language model based on an adaptive corpus (1140) containing modified texts. Each modified text of the first embodiment may include a replacement word. For example, 'Metaverse' in the first adaptive corpus (1141), 'Samsung' in the second adaptive corpus (1142), 'Memory' in the third adaptive corpus (1143), 'Visionsection' and 'Lexifeat' in the fourth adaptive corpus (1144), and 'Lexifeat' in the fifth adaptive corpus (1145) may be replacement words.

[0222] The electronic device (100) can train a language model based on the order of word combinations included in an adaptive corpus containing modified text. For example, the electronic device (100) can update the probability of acquiring the alternative word 'metaverse' based on the order of word combinations of the alternative word 'metaverse' and the remaining words 'on the platform', 'virtual character', and 'can create' in the first adaptive corpus. For example, the electronic device (100) can adjust the language model to increase the probability of predicting the word 'metaverse' in text containing the expressions 'platform' and 'virtual character'.

[0223] The electronic device (100) can train a language model based on a previously stored corpus (1150). The electronic device (100) can update statistical information regarding words, phrases, and sentence structures of the language model by analyzing text data contained within the previously stored corpus (1150).

[0224] FIG. 12 is a flowchart illustrating an operation to update a word modification table according to one embodiment of the present disclosure.

[0225] In operation 1200, the electronic device (100) may update a word modification table as it obtains a user word corresponding to at least one word to be edited included in the modified text. In one embodiment, the modified text may represent a modified text in which the word to be edited included in the text is changed to a replacement word by referring to the word modification table in operation 1010 described above with reference to FIG. 10.

[0226] The electronic device (100) can perform operation 1200 after outputting text corresponding to voice input in operation 460 described with reference to FIG. 4. After performing operation 1200, the electronic device (100) can train (or update) a function corresponding to voice recognition by performing the operation described above with reference to FIG. 10, FIG. 11a, FIG. 11b and FIG. 11c.

[0227] In one embodiment of the present disclosure, the electronic device (100) can accumulate data of word modification history by updating a word modification table before training (or updating) a function corresponding to speech recognition. In one embodiment of the present disclosure, the electronic device (100) can improve the accuracy of a function corresponding to speech recognition by training (or updating) a function corresponding to speech recognition only when a specific word in the word modification table has been changed more than a threshold number of times.

[0228] FIG. 13a is a flowchart for explaining in detail the operation of updating a word modification table and the operation of obtaining result text based on user input included in operation 1200 according to one embodiment of the present disclosure.

[0229] In operation 1210, the electronic device (100) can identify at least one word included in the modified text on a word-by-word basis. In the process of identifying at least one word on a word-by-word basis, the electronic device (100) can reflect the linguistic characteristics of the modified text. The electronic device (100) can automatically recognize the language included in the modified text or set a word identification strategy based on pre-specified language information. The electronic device (100) can identify (or distinguish) the modified text on a word-by-word basis using an artificial intelligence model such as a natural language processing algorithm.

[0230] For example, if the modified text is in Hangul, nouns or verbs may be connected to particles. The electronic device (100) identifies at least one word included in the modified text based on spacing, and can identify it as a separate word even if a noun and a particle are combined. For example, if the modified text is "to eat a sagoe," the electronic device (100) can identify the modified text by dividing it into "sagoe," "reul," and "eat." Although "sagoeul" is a single unit based on spacing, the electronic device (100) can identify it as the separate words "sagoe" and "reul" by reflecting linguistic characteristics.

[0231] The electronic device (100) can individually highlight (e.g., shade) or output words identified word by word in a clickable form.

[0232] In operation 1220, the electronic device (100) can identify a word to be edited among at least one word included in the modified text. The word to be edited may be a word in the modified text that the user wants to change. The word to be edited may be a word in the modified text that needs to be changed. For example, the electronic device (100) can acquire a user click or touch of the word to be edited among at least one word. As the electronic device (100) identifies the word to be edited, it can highlight or emphasize the word to be edited.

[0233] In operation 1230, the electronic device (100) can obtain a user word corresponding to the word to be edited. For example, the electronic device (100) can obtain a user word corresponding to the word to be edited through a keyboard, a touch screen, or a microphone. For example, the electronic device (100) can enable soft keyboard input (e.g., a virtual keyboard) on the display to support the user in directly inputting the word to be edited. For example, the electronic device (100) can enable soft keyboard input (e.g., a virtual keyboard) on the display as the user touches the word to be edited in a touch-based user interface to switch to an edit mode. The user word may be a word that the word to be edited has been modified by the user.

[0234] In operation 1240, the electronic device (100) can change the word to be edited included in the modified text to a user word. The electronic device (100) can obtain the resulting text in which the word to be edited included in the modified text is changed to a user word. For example, the electronic device (100) can identify the location of the word to be edited in the modified text and replace (or change) the word to be edited with a user word. The electronic device (100) can output the resulting text in which the word to be edited included in the modified text is changed to a user word. The electronic device (100) can store the resulting text.

[0235] In operation 1250, the electronic device (100) can update the word modification table based on the correspondence between the word to be edited and the user word. For example, the electronic device (100) can add the word to be edited to the word to be modified in the word modification table. For example, the electronic device (100) can add the user word to the replacement word in the word modification table. For example, the electronic device (100) can add the user word to the replacement word corresponding to the word to be edited added to the word to be modified in the word modification table.

[0236] As the structure of the word correction table has been explained earlier with reference to Figure 9, redundant explanations will be omitted here for the sake of brevity.

[0237] FIG. 13b is a reference diagram for explaining in detail the operation of updating a word modification table based on user input and the operation 1300 of obtaining result text according to one embodiment of the present disclosure.

[0238] In one embodiment of the present disclosure, the electronic device (100) may acquire a voice input (1310). For example, the voice input may be voice data corresponding to "Examples of Generative AI include Visionsection and Lexifeat".

[0239] In one embodiment of the present disclosure, an electronic device (100) may obtain a modified text (1320) corresponding to a voice input. The electronic device (100) may convert the voice input into text based on a function corresponding to voice recognition, and convert the text into a modified text based on a word modification table. For example, the modified text (1320) may include "Examples of Generative AI include Visionsection and Lexyfeat". The modified text (1320) may be a misrecognition of "Lexifeat" in the voice input.

[0240] In one embodiment of the present disclosure, the electronic device (100) may choose to edit the modified text (1330). For example, the electronic device (100) may choose to edit the modified text (1330) by obtaining a user action of pressing an edit button (1332). The edit button (1332) may be a physical button included in the electronic device (100) or a virtual button displayed on a screen.

[0241] In one embodiment of the present disclosure, an electronic device (100) can identify at least one word included in the modified text on a word-by-word basis (1340). For example, the electronic device (100) can identify "Examples of Generative AI include Visionsection and Lexyfeat" included in the modified text (1320) as "Examples", "of", "Generative", "AI", "include", "Visionsection", "and", "Lexyfeat".

[0242] In one embodiment of the present disclosure, an electronic device (100) can identify a word to be edited (1352) among at least one word (1350) included in the modified text. For example, the electronic device (100) can identify "Lexyfeat" as the word to be edited (1352) among the identified words "Examples", "of", "Generative", "AI", "include", "Visionsection", "and", and "Lexyfeat".

[0243] In one embodiment of the present disclosure, the electronic device (100) may obtain a user word (1362) corresponding to a word to be edited. For example, the electronic device (100) may obtain a user word "Lexifeat" corresponding to the word to be edited "Lexyfeat" among the identified words (1360).

[0244] In one embodiment of the present disclosure, an electronic device (100) can change a word to be edited included in a modified text to a user word. The electronic device (100) can obtain a result text (1370) in which the word to be edited included in the modified text is changed to a user word. For example, the electronic device (100) can change the word to be edited "Lexyfeat" to the user word "Lexifeat". For example, the electronic device (100) can obtain the result text "Examples of Generative AI include Visionsection and Lexifeat" in which the word to be edited "Lexyfeat" among the identified words is changed to the user word "Lexifeat".

[0245] In one embodiment of the present disclosure, the electronic device (100) may update (or update) the word modification table (1380) based on the correspondence between the word to be edited and the user word. For example, the electronic device (100) may add "Lexyfeat" to the word to be modified in the word modification table (1380). For example, the electronic device (100) may add "Lexifeat" to the replacement word corresponding to "Lexyfeat" in the word modification table (1380).

[0246] FIG. 14 is a flowchart illustrating the operation of updating a word modification table by comparing it with the contents of an existing word modification table according to one embodiment of the present disclosure.

[0247] In one embodiment of the present disclosure, the electronic device (100) may perform operations described with reference to FIG. 14 after operation 1230 described with reference to FIG. 13a and FIG. 13b. In operation 1230, the electronic device (100) may obtain a user word corresponding to the word to be edited.

[0248] In operation 1410, the electronic device (100) can identify (or determine, decide) whether the word to be modified included in the word modification table includes the word to be edited, and whether the replacement word corresponding to the word to be modified in the word modification table does not include the user word. For example, the electronic device (100) can compare the word to be edited and the user word with the word to be modified and the replacement word in the word modification table.

[0249] In one embodiment of the present disclosure, an electronic device (100) may retrieve a word modification table. The electronic device (100) may retrieve a word modification table stored in the internal memory of the electronic device (100). The electronic device (100) may look up (or search for, identify) the word to be modified and the replacement word included in the word modification table.

[0250] From now on, the word modification table of the electronic device (100) includes the word to be modified, ‘Bitcoin’, and the replacement word ‘Bitcoin’ corresponding to the word to be modified. The operation 1410 will be explained by taking the case where the word to be edited of the electronic device (100) is ‘Bitcoin’ and the user word corresponding to the word to be edited is ‘Bitcoin’.

[0251] In one embodiment of the present disclosure, the electronic device (100) can determine the inclusion relationship by comparing the word to be edited ('BTcoin') with the word to be modified ('BTcoin') in the word modification table. For example, if the word to be edited is included in the word to be modified, the electronic device (100) can compare the user word corresponding to the word to be edited. For example, if the word to be edited is not included in the word to be modified, the electronic device (100) can perform operation 1230 without comparing the user word corresponding to the word to be edited.

[0252] In one embodiment of the present disclosure, the electronic device (100) can determine the inclusion relationship by comparing a replacement word ('Bitcoin') corresponding to a word to be modified ('Bitcoin') with a user word ('Bitcoin') corresponding to a word to be edited. For example, if the replacement word does not include the user word, the electronic device (100) can perform operation 1420. That is, the electronic device (100) can identify that the word to be modified included in the word modification table includes the word to be edited, and that the replacement word corresponding to the word to be modified in the word modification table does not include the user word.

[0253] In one embodiment of the present disclosure, the electronic device (100) may perform operation 1240 as it does not correspond to the condition of operation 1410. For example, the electronic device (100) may perform operation 1240 if the word to be modified included in the word modification table does not include the word to be edited. For example, the electronic device (100) may perform operation 1240 if the word to be modified included in the word modification table includes the word to be edited, and the replacement word corresponding to the word to be modified in the word modification table includes the user word.

[0254] In operation 1420, the electronic device (100) can obtain user confirmation regarding whether to change the word to be edited to a user word. In one embodiment, the electronic device (100) can obtain user confirmation regarding whether to change the word to be edited to a user word as the word to be edited included in the word modification table includes the word to be edited, and the replacement word corresponding to the word to be edited in the word modification table does not include the user word.

[0255] The electronic device (100) can obtain user confirmation in various ways. For example, the electronic device (100) can output a notification screen requesting user confirmation regarding whether to change the word to be edited to a user word. For example, the electronic device (100) can output a voice message requesting user confirmation regarding whether to change the word to be edited to a user word.

[0256] For example, as in the example described in operation 1410 above, if the word to be edited is 'BTcoin' and the user word is 'Bitocoin', the electronic device (100) may output a notification screen containing a question such as "Would you like to change BTcoin to Bitcoin?". For example, the electronic device (100) may output a sound that reads out "Would you like to change BTcoin to Bitcoin?". For example, the electronic device (100) may obtain user confirmation by selecting "Yes" or "No" on the notification screen. The electronic device (100) may obtain user confirmation by obtaining an utterance that answers "Yes" or "No". The electronic device (100) may obtain user confirmation by obtaining a user gesture corresponding to "Yes" or "No" (e.g., nodding or shaking the head left and right).

[0257] In one embodiment of the present disclosure, the electronic device (100) may perform operation 1240 as it obtains user confirmation that the word to be edited is changed to a user word in operation 1420.

[0258] In operation 1430, the electronic device (100) may obtain additional user input that modifies the user word. As the electronic device (100) obtains user confirmation in operation 1420 that it does not change the word to be edited to the user word, it may obtain additional user input that modifies the user word.

[0259] The electronic device (100) can obtain additional user input in various ways. For example, the electronic device (100) can output a word to be edited and a user word corresponding to the word to be edited. For example, the electronic device (100) can output at least one alternative word (or a list of alternative words) corresponding to the word to be edited. The electronic device (100) can obtain a first additional user input for selecting one alternative word from the at least one alternative word (or a list of alternative words). Upon obtaining the first additional user input, the electronic device (100) can change the word to be edited included in the modified text to the alternative word corresponding to the first additional user input. Upon obtaining the first additional user input, the electronic device (100) can update (or update) a word modification table based on the word to be edited and the alternative word corresponding to the word to be edited.

[0260] For example, the electronic device (100) may output a word to be edited and a user word corresponding to the word to be edited. The electronic device (100) may obtain a second additional user input that modifies the word to be edited. For example, the electronic device (100) may obtain the second additional user input through a keyboard, mouse, microphone, or touch screen. Upon obtaining the second additional user input, the electronic device (100) may change the word to be edited included in the modified text to the second additional user input. Upon obtaining the second additional user input, the electronic device (100) may update (or update) a word modification table based on the word to be edited and the second additional user input corresponding to the word to be edited.

[0261] In one embodiment of the present disclosure, the electronic device (100) may perform operation 1240 after performing operation 1430.

[0262] FIG. 15 is a block diagram illustrating the configuration of an electronic device according to one embodiment of the present disclosure.

[0263] In one embodiment, the electronic device (100) may include a memory (110), a processor (120), a communication interface (130), a display (140), a microphone (150), an input / output interface (160), an audio output interface (170), a video processing interface (180), an audio processing interface (185), and a power module (190). The electronic device (100) may be composed of various combinations of the components shown in FIG. 15, and not all of the components shown in FIG. 15 are essential components.

[0264] The memory (110) can store a program related to the operation of the electronic device (100) and various data generated during the operation of the electronic device (100). The memory (110) can store at least one instruction. Additionally, the memory (110) may store at least one instruction executed by the processor (120). Additionally, the memory (110) may store at least one program executed by the processor (120). Additionally, the memory (110) may store an application for providing a specific service.

[0265] The memory (110) may include various types of memory. The memory (110) may include a main memory that stores data currently being processed in the electronic device (100). For example, the main memory may include non-volatile memory including at least one of ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), and PROM (Programmable Read-Only Memory), and volatile memory such as RAM (Random Access Memory) or SRAM (Static Random Access Memory).

[0266] The memory (110) may include a secondary memory that permanently stores a large amount of data (e.g., programs, system files, etc.). For example, the secondary memory may include a hard disk drive (HDD), a solid-state drive (SSD), an optical drive (e.g., a CD), a flash drive, etc., but is not limited thereto.

[0267] The processor (120) can control the overall operations of the electronic device (100). For example, the processor (120) can perform the functions of the electronic device (100) described in this disclosure by executing one or more instructions stored in memory (110) individually or collectively. The processor (120) may include a processing circuit. There may be one or more processors (120).

[0268] In an embodiment of the present disclosure, the processor (120) may store one or more instructions in an internally provided memory and control the operation of the electronic device (100) to be performed by executing one or more instructions stored in the internally provided memory. That is, the processor (120) may perform a predetermined operation by executing at least one instruction or program stored in an internal memory or memory (110) provided within the processor (120).

[0269] According to one embodiment, the processor (120) can perform the operation of the electronic device (100) described in the present disclosure by executing one or more instructions stored in memory (110).

[0270] The processor (120) may be composed of at least one of, for example, a Central Processing Unit (CPU), a Microprocessor, a Graphic Processing Unit (GPU), ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), an Application Processor (AP), a Neural Processing Unit (NPU), or an AI-dedicated processor designed with a hardware structure specialized for processing AI models, but is not limited thereto.

[0271] The communication interface (130) can perform data communication with other electronic devices (e.g., servers) under the control of the processor (120). Here, 'communication' may mean the operation of transmitting and / or receiving data, signals, requests, and / or commands, etc. The communication interface (130) may include a communication circuit.

[0272] The communication interface (130) can perform wired or wireless communication with at least one external device. The external device may be a server, etc. For example, the communication interface (130) may include at least one of a communication module, a communication circuit, a communication device, an input / output port, and an input / output plug for performing wired or wireless communication with at least one external device.

[0273] The communication interface (130) may include one or more modules that enable wired wireless communication between the electronic device (100) and a wireless communication system or between the electronic device (100) and a network where another electronic device is located. For example, the communication interface (130) may include a mobile communication module (132), a wireless internet module (134), a Wi-Fi communication module (136), and a Bluetooth communication module (138).

[0274] The mobile communication module (132) transmits and receives wireless signals with at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signals may include various forms of data such as voice call signals, video call call signals, or text / multimedia message transmission and reception.

[0275] The wireless internet module (134) refers to a module for wireless internet access, which may be built into or externally mounted on a device. Wireless internet technologies such as WLAN (Wireless LAN) (Wi-Fi), Wibro (Wireless broadband), WiMAX (World Interoperability for Microwave Access), and HSDPA (High Speed ​​Downlink Packet Access) may be used. Through the wireless internet module (134), the electronic device (100) can establish a Wi-Fi P2P (Peer to Peer) connection with another electronic device.

[0276] The communication interface (130) may include a short-range communication module for short-range communication. Short-range communication technologies such as Bluetooth, BLE (Bluetooth Low Energy), RFID (Radio Frequency Identification), infrared communication (IrDA, infrared Data Association), UWB (Ultra-Wideband), and ZigBee may be used. The communication interface (130) may include a Wi-Fi communication module (136) and a Bluetooth communication module (138) as short-range communication modules.

[0277] The display (140) can output a video signal to the screen of the electronic device (100) under the control of the processor (120). For example, the electronic device (100) can output content through the display (140). The display (140) can generate a driving signal by converting a video signal, data signal, OSD signal, control signal, etc. processed by the processor (120), and can display an image according to the driving signal. The display (140) may include any one of a liquid crystal display, a plasma display, an organic light emitting diode display, or an inorganic light emitting diode display. However, the present disclosure is not limited thereto, and the display (140) may include other types of displays capable of displaying content.

[0278] The microphone (150) can acquire voice input. For example, the microphone (150) can receive the voice of a user's utterance. The microphone (150) can convert the received voice input into an electrical signal and transmit it to the electronic device (100). The microphone (150) can detect the voice of a user in the space where the electronic device (100) is located. The microphone (150) may be placed at the bottom center or bottom right of the electronic device (100) to detect the voice of a user in the space where the electronic device (100) is located. The microphone (150) can transmit the acquired sensor value to the processor (120). The processor (120) can acquire ambient utterances of the electronic device by applying a predetermined processing to the acquired sensor value.

[0279] The input / output interface (160) processes input / output from outside the electronic device (100). The input / output interface (160) receives video (e.g., video, etc.), audio (e.g., voice, music, etc.), and additional information (e.g., EPG, etc.). The input / output interface (160) may include any one of an HDMI (High-Definition Multimedia Interface) port (162), a component jack (152), a PC port (153), a USB (Universal Serial Bus) port (154), an MHL (Mobile High-Definition Link), a DP (Display Port), a Thunderbolt, a VGA (Video Graphics Array) port, an RGB port, a D-SUB (D-subminiature), a DVI (Digital Visual Interface), and an audio jack. In one embodiment, the input / output interface (160) may be implemented to include a plurality of modules (e.g., USB port, HDMI port, etc.) for implementing the aforementioned input / output methods.

[0280] The electronic device (100) can be connected to external devices such as a display, camera, microphone, speaker, touchpad, and set-top box through an input / output interface (160). The input / output interface (160) may include a user input section. For example, the input / output interface (160) may include at least one of a key, a touch panel, and a pen recognition panel. The electronic device (100) may display various content or user interfaces according to user input received from at least one of the key, touch panel, and pen recognition panel. The key may include various types of keys, such as mechanical buttons or wheels, formed in various areas such as the front, side, or back of the main body exterior of the electronic device (100). The touch panel may detect the user's touch input and output a touch event value corresponding to the detected touch signal. When the touch panel is combined with a display panel to form a touch screen (not shown), the touch screen may be implemented with various types of touch sensors, such as capacitive, resistive, or piezoelectric.

[0281] The audio output interface (170) can output audio (e.g., voice, sound) input from the communication interface (130) or the input / output interface (160). Additionally, the audio output interface (170) can output audio stored in memory (110) under the control of the processor (120). The audio output interface (170) may include at least one of a speaker, a headphone output terminal, or an S / PDIF (Sony / Philips Digital Interface) output terminal, or a combination thereof.

[0282] The video processing interface (180) performs processing on video data played by the electronic device (100). The video processing interface (180) can perform various image / video processing on the video data, such as decoding, scaling, noise filtering, frame rate conversion, resolution conversion, rendering, etc.

[0283] The audio processing interface (185) performs processing on audio data played by the electronic device (100). Various processing such as decoding, amplification, and noise filtering can be performed on the audio data at the audio processing interface (185).

[0284] The power module (190) supplies power input from an external power source to the components inside the electronic device (100) that operate under the control of the processor (120). Additionally, the power module (190) can supply power output from one or more batteries located inside the electronic device (100) to the internal components under the control of the processor (120).

[0285] An electronic device according to one embodiment of the present disclosure may include at least one processor comprising a memory for storing at least one instruction and a circuit device. By having the at least one instruction executed individually or collectively by the at least one processor, the electronic device may be controlled to perform a function corresponding to voice recognition for the voice input and output text corresponding to the voice input if the acquired voice input contains at least one security word. By having the at least one instruction executed individually or collectively by the at least one processor, the electronic device may be controlled to output text corresponding to the voice input based on a function corresponding to automatic voice recognition performed on a server for the voice input if the acquired voice input does not contain at least one security word. According to one embodiment of the present disclosure, by having the at least one instruction executed individually or collectively by the at least one processor, the electronic device may acquire a similarity between the voice input and the at least one security word. According to one embodiment of the present disclosure, by executing the at least one instruction individually or collectively by the at least one processor, the electronic device can identify that the voice input contains at least one security word if the similarity is greater than or equal to a threshold similarity.

[0286] According to one embodiment of the present disclosure, by executing the at least one instruction individually or collectively by the at least one processor, the electronic device may be controlled to transmit the voice input to the server if the voice input does not contain the at least one security word. According to one embodiment of the present disclosure, by executing the at least one instruction individually or collectively by the at least one processor, the electronic device may receive from the server text corresponding to the voice input based on a function corresponding to automatic speech recognition performed on the server.

[0287] According to one embodiment of the present disclosure, by executing the at least one instruction individually or collectively by the at least one processor, the electronic device can activate a function corresponding to voice recognition for the voice input based on a trigger word included in the voice input. According to one embodiment of the present disclosure, by executing the at least one instruction individually or collectively by the at least one processor, the electronic device can control to output text corresponding to the voice input by performing a function corresponding to voice recognition using an automatic voice recognition engine for the entire voice input when the trigger word is included in the at least one security word.

[0288] According to one embodiment of the present disclosure, by executing the at least one instruction individually or collectively by the at least one processor, the electronic device can be controlled to output text corresponding to the voice input by performing a function corresponding to voice recognition using the automatic speech recognition engine for the voice input corresponding to the segment and the voice input after the segment when the at least one security word is included in the text stream accumulated in units of a predetermined segment of the voice input.

[0289] According to one embodiment of the present disclosure, by executing the at least one instruction individually or collectively by the at least one processor, the electronic device can obtain user input corresponding to the modification of the at least one security word. According to one embodiment of the present disclosure, by executing the at least one instruction individually or collectively by the at least one processor, the electronic device can identify whether the at least one security word reflecting the user input is included in the voice input.

[0290] According to one embodiment of the present disclosure, by executing the at least one instruction individually or collectively by the at least one processor, the electronic device can control the automatic speech recognition engine to train when the word to be modified included in the text is changed to the replacement word more than a threshold number of times based on a stored word modification table.

[0291] According to one embodiment of the present disclosure, by executing the at least one instruction individually or collectively by the at least one processor, the electronic device can update the probability of obtaining the replacement word from the voice input based on the characteristics of the at least one voice input containing the word to be modified.

[0292] According to one embodiment of the present disclosure, by executing the at least one instruction individually or collectively by the at least one processor, the electronic device can update the word modification table as it obtains a user word corresponding to at least one edit target word included in the modification text.

[0293] According to one embodiment of the present disclosure, by executing the at least one instruction individually or collectively by the at least one processor, the electronic device may obtain a second user input identifying the word to be edited among the at least one word. According to one embodiment of the present disclosure, by executing the at least one instruction individually or collectively by the at least one processor, the electronic device may obtain the user word corresponding to the word to be edited. According to one embodiment of the present disclosure, by executing the at least one instruction individually or collectively by the at least one processor, the electronic device may obtain the result text in which the word to be edited included in the modified text is changed to the user word.

[0294] According to one embodiment of the present disclosure, by executing the at least one instruction individually or collectively by the at least one processor, the electronic device may obtain user confirmation regarding whether to change the word to be edited to the user word if the word to be edited included in the word modification table includes the word to be edited, and the replacement word corresponding to the word to be edited in the word modification table does not include the user word.

[0295] A method of operation of an electronic device according to one embodiment of the present disclosure may include, if the acquired voice input contains at least one security word, an operation of controlling to perform a function corresponding to voice recognition for the voice input and output text corresponding to the voice input. The method may include, if the acquired voice input does not contain the at least one security word, an operation of controlling to output text corresponding to the voice input based on a function corresponding to automatic voice recognition performed on a server for the voice input.

[0296] According to one embodiment, the method may include an operation of obtaining a similarity between the voice input and the at least one security word. The method may include an operation of identifying that the voice input contains at least one security word if the similarity is greater than or equal to a threshold similarity.

[0297] According to one embodiment, the method may include an operation of controlling the transmission of the voice input to the server if the voice input does not contain the at least one security word. The method may further include an operation of receiving text corresponding to the voice input from the server based on a function corresponding to automatic speech recognition performed on the server.

[0298] According to one embodiment, the method may include an operation of activating a function corresponding to voice recognition for the voice input based on a trigger word included in the voice input. According to one embodiment, if the trigger word is included in the at least one security word, the method may include an operation of controlling to output text corresponding to the voice input by performing a function corresponding to voice recognition using a function corresponding to voice recognition for the entire voice input.

[0299] According to one embodiment, the method may include an operation of controlling the output of text corresponding to the voice input by performing a function corresponding to voice recognition using a function corresponding to voice recognition on the voice input corresponding to the section and the voice input after the section when the at least one security word is included in a text stream accumulated in units of a predetermined section of the voice input.

[0300] According to one embodiment, the method may include an operation of obtaining user input corresponding to a modification of the at least one security word. The method may include an operation of identifying whether the at least one security word reflecting the user input is included in the voice input.

[0301] According to one embodiment, the method may include an operation to control training a function corresponding to speech recognition when, based on a stored word modification table, a word to be modified included in the text is changed to the replacement word more than a threshold number of times.

[0302] One embodiment of the present disclosure may provide a computer-readable recording medium having a program recorded thereon for performing any of the above-described methods on a computer.

[0303] Some embodiments may also be implemented in the form of a recording medium containing computer-executable instructions, such as program modules executed by a computer. A computer-readable medium may be any available medium accessible by a computer and includes both volatile and non-volatile media, and both removable and non-removable media. Additionally, a computer-readable medium may include a computer storage medium. A computer storage medium includes both volatile and non-volatile, removable and non-removable media implemented by any method or technique for storing information, such as computer-readable instructions, data structures, program modules, or other data.

[0304] The disclosed embodiments may be implemented as a software program comprising instructions stored on a computer-readable storage media.

[0305] A computer is a device capable of calling instructions stored from a storage medium and performing operations according to the disclosed embodiments according to the called instructions, and may include an electronic device according to the disclosed embodiments.

[0306] Computer-readable storage media may be provided in the form of non-transitory storage media. Here, 'non-transitory' means merely that the storage medium does not contain a signal and is tangible, without distinguishing whether data is stored semi-permanently or temporarily on the storage medium.

[0307] In addition, the control method according to the disclosed embodiments may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product.

[0308] A computer program product may include a software program and a computer-readable storage medium on which the software program is stored. For example, a computer program product may include a product in the form of a software program (e.g., a downloadable app) that is electronically distributed through a device manufacturer or an electronic market (e.g., Google Play Store, App Store). For electronic distribution, at least a portion of the software program may be stored on a storage medium or temporarily created. In this case, the storage medium may be a server of the manufacturer, a server of the electronic market, or a storage medium of a relay server that temporarily stores the software program.

[0309] A computer program product may include a storage medium of a server or a storage medium of a device in a system composed of a server and a device. Alternatively, if a third device (e.g., a smartphone) is communicationally connected to the server or device, the computer program product may include a storage medium of the third device. Alternatively, the computer program product may include the S / W program itself that is transmitted from the server to the device or the third device, or transmitted from the third device to the device.

[0310] In this case, one of the server, the device, and the third device may execute the computer program product to perform the method according to the disclosed embodiments. Alternatively, two or more of the server, the device, and the third device may execute the computer program product to perform the method according to the disclosed embodiments in a distributed manner.

[0311] For example, a server (e.g., a cloud server or an artificial intelligence server, etc.) can execute a computer program product stored on the server to control a device connected to the server in communication to perform a method according to the disclosed embodiments.

[0312] As another example, the third device may execute a computer program product to control a device connected to the third device in communication to perform a method according to the disclosed embodiment. When the third device executes the computer program product, the third device may download the computer program product from a server and execute the downloaded computer program product. Alternatively, the third device may execute a computer program product provided in a preloaded state to perform a method according to the disclosed embodiments.

[0313] Additionally, in this specification, "interface" may be a hardware component, such as a processor or circuit, and / or a software component executed by a hardware component, such as a processor.

[0314] The foregoing description of the present disclosure is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the present disclosure. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0315] The scope of the present disclosure is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the present disclosure.

Claims

1. In an electronic device (100), A memory (110) that stores at least one instruction and It includes at least one processor (120) including a circuit device, and By having the above at least one instruction executed individually or collectively by the above at least one processor (120), the electronic device (100) If the acquired voice input contains at least one security word, control is performed to perform a function corresponding to voice recognition for the voice input and output text corresponding to the voice input, and An electronic device (100) that controls to output text corresponding to the voice input based on a function corresponding to automatic voice recognition performed on a server for the voice input if the voice input obtained above does not contain at least one security word.

2. In Paragraph 1, By having the above at least one instruction executed individually or collectively by the above at least one processor (120), the electronic device (100) Obtaining the similarity between the above voice input and the above at least one security word, and An electronic device (100) that identifies that the voice input contains at least one security word if the similarity is greater than or equal to a threshold similarity.

3. In Paragraph 1 or 2, By having the above at least one instruction executed individually or collectively by the above at least one processor (120), the electronic device (100) If the above voice input does not include at least one security word, control to transmit the voice input to the server, and An electronic device (100) that receives text corresponding to the voice input from the server based on a function corresponding to automatic voice recognition performed on the server.

4. In any one of paragraphs 1 through 3, By having the above at least one instruction executed individually or collectively by the above at least one processor (120), the electronic device (100) Based on the trigger word included in the above voice input, a function corresponding to voice recognition for the above voice input is activated, and An electronic device (100) that controls to output text corresponding to the voice input by performing a function corresponding to voice recognition using a function corresponding to voice recognition for the entire voice input when the above trigger word is included in the above at least one security word.

5. In any one of paragraphs 1 through 4, By having the above at least one instruction executed individually or collectively by the above at least one processor (120), the electronic device (100) An electronic device (100) that controls to output text corresponding to the voice input by performing a function corresponding to voice recognition using a function corresponding to voice recognition for the voice input corresponding to the section and the voice input after the section when at least one security word is included in a text stream accumulated in a predetermined section unit of the voice input.

6. In any one of paragraphs 1 through 5, By having the above at least one instruction executed individually or collectively by the above at least one processor (120), the electronic device (100) Obtaining user input corresponding to the modification of at least one security word, and An electronic device (100) for identifying whether the at least one security word reflecting the above user input is included in the voice input.

7. In any one of paragraphs 1 through 6, By having the above at least one instruction executed individually or collectively by the above at least one processor (120), the electronic device (100) An electronic device (100) that controls training a function corresponding to speech recognition when a word to be modified included in the text is changed to a replacement word more than a threshold number of times based on a stored word modification table.

8. In Paragraph 7, By having the above at least one instruction executed individually or collectively by the above at least one processor (120), the electronic device (100) An electronic device (100) that updates the probability of obtaining the replacement word from the voice input based on the characteristics of at least one voice input containing the word to be modified.

9. In Paragraph 8, By having the above at least one instruction executed individually or collectively by the above at least one processor (120), the electronic device (100) An electronic device (100) that updates the probability of obtaining the replacement word from any text based on the combination order of at least one word included in the modified text in which the modified word included in the text is changed to the replacement word.

10. In Paragraph 9, By having the above at least one instruction executed individually or collectively by the above at least one processor (120), the electronic device (100) An electronic device (100) that updates the word modification table by obtaining a user word corresponding to at least one edit target word included in the above modification text.

11. In a method of operating an electronic device, If the acquired voice input contains at least one security word, an operation to perform a function corresponding to voice recognition for the voice input and control to output text corresponding to the voice input; and A method comprising controlling to output text corresponding to the voice input based on a function corresponding to automatic speech recognition performed on a server for the voice input, if the voice input obtained above does not contain at least one security word.

12. In Paragraph 11, The operation of identifying whether the above voice input contains at least one security word is, An operation to obtain similarity between the above voice input and the at least one security word; and A method comprising an action of identifying that the voice input contains at least one security word if the similarity is greater than or equal to a threshold similarity.

13. In Paragraph 11 or 12, If the above voice input does not contain the at least one security word, an operation to control the voice input to be transmitted to the server; and A method further comprising the operation of receiving text corresponding to the voice input from the server based on a function corresponding to automatic voice recognition performed on the server.

14. In any one of paragraphs 11 through 13, If the above-mentioned acquired voice input contains at least one security word, the operation of controlling to perform a function corresponding to voice recognition for the voice input and output text corresponding to the voice input is, An operation to activate a function corresponding to voice recognition for the voice input based on a trigger word included in the voice input; and A method comprising, when the trigger word is included in the at least one security word, controlling to output text corresponding to the voice input by performing a function corresponding to voice recognition using a function corresponding to voice recognition for the entire voice input.

15. A computer-readable recording medium having a program recorded thereon for performing the method of any one of paragraphs 11 through 14 on a computer.

Images