Electronic device and method for providing or obtaining data for training the electronic device
By collecting and sending voice data and recognition results to the second electronic device in the first electronic device, the problem of insufficient performance of the existing voice recognition model in the actual environment is solved, and effective update of the voice recognition model and multiple devices are realized. Co-training improves recognition accuracy.
Patent Information
- Application Number
- CN201980044694.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2019-02-01
- Filing Date
- 2019-09-20
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2039-09-20
AI Technical Summary
Existing speech recognition models do not provide optimal performance in real-world environments with noisy or obstructions, and a method is needed to update the model to adapt to the data of the actual environment.
User voice data is collected by the first electronic device and corresponding voice recognition results are obtained, and the data is then sent to the second electronic device for training the voice recognition model. Meanwhile, the second electronic device may also receive voice data and recognition results from the first electronic device to update its own voice recognition model.
This method allows effective update of the speech recognition model in a practical environment, improves recognition accuracy in the presence of noisy or obstacles, and can be used for speech recognition model training shared by multiple electronic devices.
Smart Images

Figure CN112384974B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to an electronic device and an operating method thereof. More specifically, the present disclosure relates to a method for providing or obtaining data for training. Background Art
[0002] Research has been conducted to improve the accuracy of speech recognition. In particular, research on acoustic models and language models that form speech recognition models has been actively developed. The acoustic model and language model can be updated through training. In this case, accurate data for training the correct answer may be needed to improve speech recognition.
[0003] The speech recognition engine can be implemented as a neural network model using an artificial intelligence (AI) algorithm. Unlike existing rule-based and statistical intelligent systems, artificial intelligence (AI) systems are computer systems that implement human-level intelligence that learns, judges, and becomes intelligent. As the use of AI systems improves, the recognition rate and the understanding or anticipation of user tastes can be performed more accurately. Therefore, existing rule-based intelligent systems are gradually being replaced by deep learning-based AI systems.
[0004] Artificial intelligence technology consists of machine learning (deep learning) and basic technologies that utilize machine learning. Machine learning is an algorithmic technology that can classify or learn the characteristics of input data. Elementary technology is a technology that uses machine learning algorithms such as deep learning to simulate functions such as recognition and judgment of the human brain. Machine learning consists of technical fields such as language understanding, visual understanding, reasoning, prediction, knowledge representation, motion control, etc.
[0005] Examples of various fields where AI technology is applied are described below. Language understanding is a technology for recognizing, applying and / or processing human language or characters, and includes natural language processing, machine translation, dialogue systems, question and answer, speech recognition or synthesis, etc. Visual understanding is a technology for recognizing and processing objects as human vision, including object recognition, object tracking, image search, human recognition, scene understanding, spatial understanding, image enhancement, etc. Inference prediction is a technology for judging and logically inferring and predicting information, including knowledge-based and probability-based inference, optimization prediction, preference-based planning, recommendation, etc. Knowledge representation is a technology for automating human experience information into knowledge data, including knowledge construction (data generation or classification), knowledge management (data utilization), etc. Motion control is a technology for controlling the autonomous driving of vehicles and the movement of robots, including motion control (navigation, collision, driving), operation control (behavior control), etc.
[0006] The above information is presented as background information only to assist with an understanding of the present disclosure. No determination has been made, and no assertion is made, as to whether any of the above may be applicable as prior art with respect to the present disclosure. Summary of the invention
[0007] Technical issues
[0008] Generally, a voice recognition model for recognizing voice input via an electronic device can be generated using data for training collected in an optimal environment. For example, a voice recognition model can be trained using data for training collected in an optimal experimental environment that is well soundproofed or not blocked by obstacles.
[0009] However, there may be various variables in the environment where the actual speech recognition model is used. For example, the environment in which the user speaks to the electronic device may be noisy, and the user may speak to the electronic device from a long distance. Alternatively, there may be obstacles between the user and the electronic device.
[0010] In this case, the speech recognition model trained in the optimal environment may not provide the best performance in the actual speech environment. Therefore, the speech recognition model needs to be updated to take into account the actual speech environment, and therefore the speech recognition model may need to be updated with the best data for training taking into account the actual speech environment.
[0011] If there are multiple electronic devices that can respond to the user's speech, each voice recognition model may need data for training in order to train the voice recognition models used by the multiple electronic devices. Therefore, it may be necessary to use a method that can represent data for training that can quickly and effectively update the corresponding voice recognition models.
[0012] Aspects of the present disclosure are to address at least the above-mentioned problems and / or disadvantages and to provide at least the advantages described below. Additional aspects will be set forth in part in the description below and in part will be apparent from the description, or may be learned by practicing the presented embodiments.
[0013] Technical Solution
[0014] According to one aspect of the present disclosure, a method for providing data for training by a first electronic device is provided. The method includes obtaining first voice data of a voice uttered by a user at a specific time through a microphone of the first electronic device, obtaining a voice recognition result corresponding to the first voice data, and sending the voice recognition result to a second electronic device that obtains second voice data of a voice uttered by the user at the specific time, so as to be used as data for training a voice recognition model.
[0015] According to another aspect of the present disclosure, a method for obtaining data for training by a second electronic device is provided. The method includes obtaining second voice data of a voice uttered by a user at a specific time through a microphone of the second electronic device, receiving a voice recognition result corresponding to the first voice data from a first electronic device that obtains first voice data of a voice uttered by the user at the specific time, and using the second voice data and the voice recognition result corresponding to the first voice data as data for training a voice recognition model.
[0016] According to another aspect of the present disclosure, a first electronic device is provided. The first electronic device includes: a microphone configured to receive a voice emitted by a user; a communicator configured to communicate with a second electronic device; at least one processor configured to execute at least one instruction; and a memory configured to store the at least one instruction. The at least one processor can perform the following operations based on the execution of the at least one instruction: obtain first voice data of a voice emitted by a user at a specific time through the microphone, obtain a voice recognition result corresponding to the first voice data, and send the voice recognition result to a second electronic device that obtains second voice data of a voice emitted by the user at a specific time, so as to be used as data for training a voice recognition model.
[0017] According to another aspect of the present disclosure, a second electronic device is provided. The second electronic device includes: a microphone configured to receive a voice emitted by a user; a communicator configured to communicate with a first electronic device; at least one processor configured to execute at least one instruction; and a memory configured to store the at least one instruction. The at least one processor can perform the following operations based on the execution of the at least one instruction: obtain second voice data of a voice emitted by a user at a specific time through the microphone, receive a voice recognition result corresponding to the first voice data from the first electronic device that obtains the first voice data of a voice emitted by the user at a specific time through the communicator, and use the second voice data and the voice recognition result corresponding to the first voice data as data for training a voice recognition model.
[0018] According to the present disclosure, a computer program product may include a non-transitory computer-readable storage medium, which includes instructions, which are executed by at least one processor of a first electronic device based on the instructions, so that the first electronic device performs the following operations: obtaining first voice data of a voice uttered by a user at a specific time through a microphone of the first electronic device, obtaining a voice recognition result corresponding to the first voice data, and sending the voice recognition result to a second electronic device that obtains second voice data of a voice uttered by the user at the specific time, so as to be used as data for training a voice recognition model.
[0019] According to the computer program product of the present disclosure, a non-transitory computer-readable storage medium includes instructions, which are executed by at least one processor of a second electronic device based on the instructions, so that the second electronic device performs the following operations: obtain second voice data of a voice uttered by a user at a specific time through a microphone of the second electronic device, receive a voice recognition result corresponding to the first voice data from a first electronic device that obtains first voice data of a voice uttered by the user at the specific time, and use the second voice data and the voice recognition result corresponding to the first voice data as data for training the voice recognition result.
[0020] Beneficial Effects
[0021] According to the present disclosure, a correct answer (e.g., a speech recognition result) corresponding to a speech uttered by a user can be obtained using an electronic device (e.g., a first electronic device) used as a representative. In this case, a speech recognition model used by another electronic device (e.g., a second electronic device) can be effectively updated (or trained) by using the speech data of the user's speech and the speech recognition result of the user's speech as data for training. In addition, the data for training can be used to train a speech recognition model used together by each of a plurality of other electronic devices (e.g., a second electronic device and a third electronic device).
[0022] In addition, the place where the user's speech frequently occurs can be determined to some extent (e.g., on the sofa, etc.). In this case, the electronic device located closest to the user (e.g., the first electronic device) can more accurately recognize the user's voice and obtain a correct voice recognition result corresponding to the user's voice compared to other electronic devices located at a long distance (e.g., the second electronic device or the third electronic device).
[0023] In this case, if an electronic device located near the user shares the voice recognition result obtained by the user with other electronic devices located remotely, the other electronic devices located remotely can obtain the correct voice recognition result corresponding to the voice recognized from a distance. If the other electronic devices use the voice recognition result as data for training, the voice recognition model used by each other electronic device can be effectively trained.
[0024] In addition, various effects directly or indirectly grasped can be provided by the present disclosure.
[0025] Other aspects, advantages, and salient features of the present disclosure will become apparent to those skilled in the art from the following detailed description, which, taken in conjunction with the accompanying drawings, discloses various embodiments of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent through the following description in conjunction with the accompanying drawings, in which:
[0027] FIG1A is a diagram of a system according to an embodiment of the present disclosure;
[0028] FIG. 1B is a diagram of a system without considering a first electronic device according to an embodiment of the present disclosure;
[0029] Figure 2 and Figure 3 is a diagram of a system according to another embodiment of the present disclosure;
[0030] Figure 4 is a block diagram of a configuration of an electronic device according to an embodiment of the present disclosure;
[0031] Figure 5 is a diagram illustrating voices input to a first electronic device and a second electronic device according to an embodiment of the present disclosure;
[0032] Figure 6 is a flow chart between a first electronic device and a second electronic device according to an embodiment of the present disclosure;
[0033] Figure 7 is a flow chart of a first electronic device according to an embodiment of the present disclosure;
[0034] Figure 8 is a flow chart of a second electronic device according to an embodiment of the present disclosure;
[0035] Fig. 9 is a detailed block diagram of an electronic device in a network environment according to an embodiment of the present disclosure;
[0036] Fig.10 is a diagram showing a system in the presence of a plurality of speakers according to an embodiment of the present disclosure; and
[0037] 11A and 11B are diagrams of a system that considers a discourse environment according to an embodiment of the present disclosure.
[0038] Throughout the drawings, like reference numerals will be understood to refer to like parts, components and structures. DETAILED DESCRIPTION
[0039] The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of the various embodiments of the present disclosure as defined by the claims and their equivalents. It includes various specific details to assist in understanding, but these details are considered to be exemplary only. Therefore, it will be appreciated by those of ordinary skill in the art that various changes and modifications may be made to the various embodiments described herein without departing from the scope and spirit of the present disclosure. In addition, descriptions of well-known functions and structures may be omitted for clarity and conciseness.
[0040] The terms and words used in the following description and claims are not limited to the bibliographical meanings, but are merely used by the inventor to enable a clear and consistent understanding of the present disclosure. Therefore, it will be apparent to those skilled in the art that the following description of various embodiments of the present disclosure is provided for illustrative purposes only and not for the purpose of limiting the present disclosure as defined by the appended claims and their equivalents.
[0041] It should be understood that the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to "a component surface" includes reference to one or more of such surfaces.
[0042] The terms used herein may be used to describe various elements, but the elements should not be limited by the terms. These terms are only used to distinguish one element from another.
[0043] Terms such as "first" and "second" used in various embodiments can modify various elements, but do not limit the corresponding elements. For example, the terms do not limit the order and / or importance of the corresponding elements. The terms can be used to distinguish one element from another element.
[0044] For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element, without departing from the scope of various embodiments of the present disclosure.
[0045] In this specification, when a component is "connected" to another component, it is not only a case where a component is "directly connected" but also a case where it is "electrically connected" to another component by inserting another component therebetween. In addition, it should be noted that any part "connected" to another part includes some of the parts being in a state capable of performing data communication via signal transmission and reception with the other part.
[0046] Some embodiments of the present disclosure can be represented by function block configuration and various processing steps. Some or all of these function blocks can be implemented with various numbers of hardware and / or software configurations that perform specific functions. For example, the function blocks of the present disclosure can be implemented by one or more microprocessors, or can be implemented by circuit configurations for given functions. In addition, for example, the function blocks of the present disclosure can be implemented in various programming or scripting languages. The function blocks can be implemented as algorithms executed in one or more processors. In addition, the present disclosure can adopt technologies for electronic environment settings, signal processing and / or data processing. The terms "element", "device" and "configuration" can be widely used and are not limited to mechanical and physical configurations.
[0047] In addition, the connecting lines or connecting members between the components shown in the drawings are functional connections and / or physical or circuit connections. In actual devices, the connections between the elements may be represented by various functional connections, physical connections or circuit connections that may be added or replaced.
[0048] In an embodiment, the term "speaker" may refer to a person who utters any speech, and the term "listener" may refer to a person who hears the uttered speech. In a conversation of multiple people, the speaker and the listener change from time to time, and there may be confusion when distinguishing each person as a speaker and a listener. Therefore, in the present disclosure, based on the language used by each person, a person using a first language may be referred to as a "first speaker", and a person using a second language may be referred to as a "second speaker".
[0049] The term "text" in the embodiments described in the present disclosure may refer to data intended to convey meaning to another party. "Text" may include data in natural language formats such as characters, symbols, words, phrases, sentences, schematic diagrams, charts, etc., data in the form of an arrangement of artificial language characters, or audio data. For example, "text" may refer to speech uttered by a speaker represented as text or voice data.
[0050] Any instance of the term "speech recognition" in the present disclosure may be replaced with the term "voice recognition", so that any aspect of the present disclosure described with reference to speech recognition may be equally applicable to voice recognition. In addition, any aspect of the present disclosure described with reference to speech recognition may be equally applicable to a combination of speech recognition and voice recognition. In addition, any instance of the term "voice recognition" in the present disclosure may be replaced with the term "speech recognition", so that any aspect of the present disclosure described with reference to voice recognition may be equally applicable to speech recognition. In addition, any aspect of the present disclosure described with reference to voice recognition may be equally applicable to a combination of speech recognition and voice recognition.
[0051] Hereinafter, the present disclosure will be further described with reference to the accompanying drawings.
[0052] FIG. 1A is a diagram of a system according to an embodiment of the present disclosure.
[0053] The system 1 of FIG. 1A may include a first electronic device 10 , a second electronic device 20 , a third electronic device 30 , a first server 50 , and a second server 60 .
[0054] The first electronic device 10 to the third electronic device 30 (for example, electronic devices according to various embodiments of the present disclosure) may include, for example, at least one of a smart phone, a tablet PC, a mobile phone, a video phone, an e-book reader, a desktop PC, a laptop PC, a netbook computer, a workstation, a server, a personal digital assistant (PDA), a portable multimedia player (PMP), an MP3 player, a medical device, a camera, or a wearable device. The wearable device may include at least one of the following: an accessory type (for example, a watch, a ring, a bracelet, a wristband, a necklace, glasses, contact lenses, or a head-mounted device (HMD)); a fabric or clothing embedded type (for example, electronic clothing); a body attachment type (for example, a skin pad or a tattoo); or a bio-implantable circuit. In some embodiments, the electronic device may include, for example, a television, a digital video disk (DVD) player, an audio device, a refrigerator, a vacuum cleaner, an oven, a microwave oven, a washing machine, an air purifier, a set-top box, a home automation control panel, a security control panel, a media box (for example: Samsung HomeSync TM , Apple TV TM or Google TV TM ), game consoles (e.g., Xbox TM , PlayStation TM ), at least one of an electronic dictionary, an electronic key, a camera, or an electronic photo frame.
[0055] In other embodiments, the electronic device may include at least one of various medical devices (e.g., various portable medical measuring devices, such as blood glucose meters, heart rate monitors, blood pressure monitors, or temperature measuring devices), magnetic resonance angiography (MRA), magnetic resonance imaging (MRI), computed tomography (CT), photographic devices, or ultrasound devices, etc.), navigation systems, global navigation satellite systems (GNSS), event data recorders (EDRs), flight data recorders (FDRs), automotive infotainment devices, marine electronic devices (e.g., marine navigation devices, gyrocompasses, etc.), avionics devices, security devices, automotive head units, industrial or household robots, drones, automated teller machines (ATMs) at financial institutions, store points of sale, or Internet of Things (IoT) devices (e.g., light bulbs, sensors, sprinkler devices, fire alarms, thermostats, street lights, toasters, exercise equipment, hot water tanks, heaters, boilers, etc.), etc.
[0056] In another embodiment, the electronic device may include at least one of furniture, a part of a building / structure or a vehicle, an electronic board, an electronic signature receiving device, a projector, or various metering instruments (e.g., a water, electricity, gas, or radio wave measuring device, etc.). The electronic device according to an embodiment of the present disclosure is not limited to the above-mentioned devices. In the present disclosure, the term user may refer to a person using an electronic device or a device using an electronic device (example: an artificial intelligence electronic device).
[0057] Referring to FIG. 1A , a user 1 (or speaker) is shown. User 1 may make a voice at a specific time (e.g., at a specific time or during a specific time). For example, user 1 may make a voice 2 "What time is it now?" In this case, a first electronic device 10 located at a first distance (e.g., a short distance) from user 1 may obtain first voice data (①) of a voice 2 made by user 1 at a specific time through a microphone provided in the first electronic device 10. The first voice data may be, for example, a digital signal obtained by converting a voice as an analog signal made by user 1 through a converter. At this time, a second electronic device 20 located at a second distance (e.g., an intermediate distance) from user 1 may also obtain second voice data (②) of a voice 2 made by user 1 at a specific time through a microphone provided in the second electronic device 20. In addition, a third electronic device 30 located at a third distance (e.g., a long distance) from user 1 may also obtain third voice data (③) of a voice 2 made by user 1 at a specific time through a microphone provided in the third electronic device 30.
[0058] In this case, the first voice data, the second voice data, and the third voice data may be different from each other. For example, the specifications of the microphones provided in the first electronic device 10, the second electronic device 20, and the third electronic device 30 are different from each other, and therefore, the voice data may be different from each other. In addition, the position of each of the first electronic device 10, the second electronic device 20, and the third electronic device 30 may be different from each other. For example, in FIG. 1A, the position at which the second electronic device 20 obtains voice 2 is farther than the position at which the first electronic device 10 obtains voice 2, and therefore, when compared with the first voice data, the second voice data obtained by the second electronic device 20 may include a time difference or phase difference, a smaller amplitude, or more noise. Similarly, when compared with the first voice data, the position at which the third electronic device 30 obtains voice 2 may be farther than the position at which the second electronic device 20 obtains voice 2, and the third voice data obtained by the third electronic device 30 may include more time differences or phase differences, a smaller amplitude, or more noise.
[0059] 1A , when the first electronic device 10 obtains the first voice data, the first electronic device 10 may obtain a voice recognition result corresponding to the obtained first voice data. For example, the first electronic device 10 may apply the voice recognition result corresponding to the obtained first voice data to a speech recognition model provided in the first electronic device 10, and obtain a voice recognition result. Optionally, the first electronic device 10 may send the first voice data to a first server 50 that can communicate with the first electronic device 10 (④). The first electronic device 10 may obtain a voice recognition result corresponding to the first voice data from the first server 50 (⑤).
[0060] Specifically, the first electronic device 10 can obtain text data as a result of the speech recognition result by applying the obtained first voice data to the speech recognition model provided in the first electronic device 10 or in the first server 50. The acoustic model may include information related to the utterance, and the language model may include information about unit phoneme information and a combination of unit phoneme information. The speech recognition module may convert the speech of the user 1 into text data using the information related to the utterance and the information about the unit phoneme.
[0061] As another example, the first electronic device 10 may apply the obtained first voice data to a voice recognition model and a natural language understanding module provided in the first electronic device 10 or the first server 50, and obtain the user's intention as a voice recognition result.
[0062] As an embodiment, the natural language understanding module can identify the user's intention by performing grammatical analysis or semantic analysis. Syntactic analysis can divide the user input into grammatical units (e.g., words, phrases, morphemes, etc.), and can grasp which grammatical elements the divided units may have. Semantic analysis can be performed using semantic matching, rule matching, formula matching, etc. Therefore, the natural language understanding module can obtain the domain, intent, or parameter (or slot) used to express the intention of the user input.
[0063] As another embodiment, the natural language understanding module can determine the user intent and parameters using matching rules divided into domains, intents, and parameters (or slots) for grasping the intent. For example, a domain (e.g., alarm) may include multiple intents (e.g., alarm setting, alarm cancellation, etc.), and an intent may include multiple parameters (e.g., time, repeat time, alarm sound, etc.). Multiple rules may include, for example, one or more mandatory element parameters. The matching rules may be stored in a natural language understanding database (NLU DB).
[0064] As another embodiment, the natural language understanding module may use language features (e.g., grammatical elements) such as morphemes or phrases to grasp the meaning of words extracted from user input, and determine the user's intent by matching the grasped meaning with the domain and intent. For example, the natural language understanding module may determine the user's intent by calculating how many words extracted from the user input are included in each domain and intent. According to one embodiment, the natural language understanding module may determine the parameters of the user's input using words that form the basis for understanding the intent. According to one embodiment, the natural language understanding module may determine the user's intent using a natural language recognition database that stores language features for grasping the intent of the user's input.
[0065] As another embodiment, the natural language module may use a personal language model (PLM) to determine the user's intent. For example, the natural language understanding module may use personalized information (e.g., contact list, music list) to determine the user's intent. The personalized language model may be stored in, for example, a natural language recognition database.
[0066] As another embodiment, the first electronic device 10 may apply the obtained first voice data to a voice recognition model and a natural language understanding module provided in the first electronic device 10 or the first server 50, and obtain a path rule as a voice recognition result.
[0067] As an embodiment, the natural language understanding module may generate a path rule based on the intent and parameters input by the user. For example, the natural language understanding module may select an application to be executed based on the intent input by the user, and determine the action to be performed in the selected application. The natural language understanding module may generate a path rule by determining parameters corresponding to the determined action. According to one embodiment, the path rule generated by the natural language understanding module may include the application to be executed, the action to be performed in the application (e.g., at least one state), and information about the parameters for performing the action.
[0068] As another embodiment, the natural language understanding module may generate one or more path rules based on the intent and parameters input by the user. For example, the natural language understanding module may receive a path rule set corresponding to the first electronic device 10 from the channel planner module, and determine the path rule by mapping the intent and parameters input by the user to the received path rule set.
[0069] As another embodiment, the natural language understanding module may generate one or more path rules by determining the application to be executed based on the intent and parameters input by the user, the action to be executed in the application, and the parameters for executing the action. For example, the natural language understanding module may generate a path rule by arranging the application to be executed or the action to be executed by the application in an ontology or a graphical model according to the intent input by the user using information of the first electronic device 10.
[0070] As another embodiment, the natural language understanding module may select at least one path rule from a plurality of generated path rules. For example, the natural language understanding module may select the best path rule from the plurality of path rules. For example, when only some actions are specified based on the user's speech, the natural language understanding module may select a plurality of path rules. The natural language understanding module may determine a path rule from the plurality of path rules through additional input from the user.
[0071] The first electronic device 10 may display on the screen voice recognition information 101 associated with the voice recognition result corresponding to the obtained first voice data. The first electronic device 10 may send the voice recognition result corresponding to the obtained first voice data (e.g., text data, user's intention, or path rule) to the second electronic device 20 (⑥) located at a second distance from the user 1 that is relatively farther from the user 1 than the position of the first electronic device 10. In addition, the first electronic device 10 may send the obtained voice recognition result to the third electronic device 30 (⑦) located at a third distance from the user 1 that is farther from the user 1 than the position of the second electronic device 20.
[0072] The second electronic device 20 or the third electronic device 30 that obtains the speech recognition result corresponding to the first speech data associated with the speech 2 uttered by the user 1 can use the speech recognition result as part of the training data (or learning labels, similar correct answers, etc.) for training the speech recognition model.
[0073] The second electronic device 20 may form a set of the second voice data of the voice uttered by the user 1 obtained in operation ② and the voice recognition result corresponding to the first voice data received in operation ⑥, and use the set as data for training. For example, when the voice recognition model is located in the second electronic device 20, the second electronic device 20 may update the voice recognition model (⑧) by applying {second voice data, voice recognition result obtained from the first electronic device 10} as data for training to the voice recognition model.
[0074] The third electronic device 30 may form a set of the third voice data of the voice uttered by the user 1 obtained in operation ③ and the voice recognition result corresponding to the first voice data obtained in operation ⑦, and use the set as data for training. For example, when the voice recognition model is located in the second server 60, the third electronic device 30 may send {third voice data, voice recognition result obtained from the first electronic device 10} as data for training to the second server 60 where the voice recognition model is located (⑨). The second server 60 may apply the received data for training to the voice recognition model and update the voice recognition model stored in the second server 60 (⑩).
[0075] According to the present disclosure, the second electronic device 20 or the third electronic device 30 may use the voice data obtained by the second electronic device 20 or the third electronic device 30 and the voice recognition result obtained by the first electronic device 10 as data for training. In this case, the second electronic device 20 or the third electronic device 30 may determine whether the voice data and the voice recognition result are suitable for use as data for training, and if it is determined that the data is suitable for data for training, the voice data and the voice recognition result may be used as data for training.
[0076] For example, in the case where the voice uttered by the user 1 is a voice related to a function of the first electronic device 10 (e.g., a call function, a camera function, etc.), the second electronic device 20 may obtain voice data about the voice through a microphone provided in the second electronic device 20, and obtain a voice recognition result corresponding to the voice from the first electronic device 10. In this case, if the second electronic device 20 does not support the function, the second electronic device 20 may ignore the voice data and the voice recognition result corresponding to the voice, and not use them as data for training. To this end, the first electronic device 10 may also send information about a target function (or target application) using the voice recognition result together with the voice recognition result corresponding to the voice to the second electronic device 20. The second electronic device 20 may determine whether the second electronic device 20 supports the information about the target function, and if it is determined that the second electronic device 20 does not support the function, the second electronic device 20 may ignore the voice data about the voice and the voice recognition result corresponding to the voice, and not use them as data for training.
[0077] The second electronic device 20 or the second server 60 may use the above-mentioned data for training to train a learning network model (or data recognition model) according to, for example, a supervised learning method based on an AI algorithm or an unsupervised learning method. The learning network model may be, for example, a model based on a neural network. The learning network model may include a plurality of network nodes with weights. In a deep learning model, a plurality of network nodes are located at different depths (or layers), and data may be sent and received according to a convolutional connection relationship. Examples of learning determination models include, but are not limited to, deep neural networks (DNNs), recursive neural networks (RNNs), and bidirectional recursive deep neural networks (BRDNNs).
[0078] The data learning module for training data provided in the second server 60 or the second electronic device 20 may be implemented as a software module or at least one hardware chip form. The data learning module may be manufactured in the form of a dedicated hardware chip or a general-purpose processor (e.g., a CPU or an application processor) or only a graphics processor (e.g., a GPU) for AI, and may be installed on the above-mentioned various electronic devices. Here, the dedicated hardware chip for AI may be a dedicated processor for probability calculation, and may have higher parallel processing performance than a general-purpose processor of the related art, so it can quickly process computing tasks in AI, such as machine learning.
[0079] As an embodiment, when there are a plurality of pre-built learning network models, the data learning module may determine a learning network model having a large correlation with the input data for training as the learning network model to be trained.
[0080] As an embodiment, the data learning module may also use a learning algorithm including, for example, an error back propagation method or a gradient descent to train the learning network model.
[0081] As an embodiment, the data learning module may train, for example, a learning network model by supervised learning using the data for training as an input value. Alternatively, the data learning module may train the learning network model by unsupervised learning to detect the standard by self-learning without specific supervision. The data learning module may train the learning network model by reinforcement learning using, for example, feedback on whether the learning result is correct.
[0082] As an embodiment, when there is a basic learning network model pre-trained in the second electronic device 20 or the second server 60, the second electronic device 20 or the second server 60 can use the above learning data to update the basic learning network model. Therefore, a learning network model optimized and adapted to the environment in which the second electronic device 20 is located is generated, and thus, the speech recognition result of the second electronic device 20 can be significantly improved.
[0083] FIG. 1B is a diagram of a system without considering the first electronic device 10 according to an embodiment of the present disclosure.
[0084] 1B , the speech recognition model used by the second electronic device 20 may be a speech recognition model updated using {second speech data, speech recognition result obtained from the first electronic device 10} as data for training as described above, and the speech recognition model used by the third electronic device 30 may be a speech recognition model updated using {third speech data, speech recognition result obtained from the first electronic device 10} as data for training as described in FIG. 1A .
[0085] 1B , user 1 may utter voice 3 at a time different from the specific time at which voice 2 is uttered. For example, voice 3 may have the same or similar pronunciation, phoneme, meaning, etc. as voice 2. For example, if voice 2 at a specific time is "What time is it now?", voice 3 uttered at a different time may be, for example, "What time is it now?", "What time is it now?", "What time?", etc. As an embodiment, after uttering a trigger voice (e.g., "refrigerator", "speaker", "Bixby", etc.) for activating a voice recognition function of at least one of the second electronic device 20 and the third electronic device 30, voice 3 may be uttered.
[0086] As an example, when the second electronic device 20 is triggered in response to the voice 3 of the user 1, the second electronic device 20 may obtain fourth voice data about the voice 3 through the microphone. When the fourth voice data is obtained, the second electronic device 20 may apply the fourth voice data to the voice recognition model updated in operation ⑧, and obtain a voice recognition result corresponding to voice 3 The second electronic device 20 may output voice recognition information 102 related to the obtained voice recognition result. For example, the voice recognition information 102 may be "the time is ~ o'clock" as a reply to the uttered voice 3.
[0087] As another embodiment, when the third electronic device 30 is triggered in response to the voice 3 of the user 1, the third electronic device 30 may obtain fifth voice data about the voice 3 emitted through the microphone. The third electronic device 30 may transmit the obtained fifth voice data to the second server 60 that can communicate with the third electronic device 30. The third electronic device 30 may obtain the speech recognition result corresponding to the fifth speech data from the second server 60 The third electronic device 30 may display voice recognition information 103 related to the obtained voice recognition result on the screen.
[0088] Therefore, generally, the second electronic device 20 or the third electronic device 30 located far away from the user 1 has the problem of difficulty in accurately recognizing the voice uttered by the user 1, but according to the present disclosure, the accurate voice recognition results and the voice of the user 1 at a long distance can be used to train the voice recognition model, so that even for the voice uttered by the user 1 at a long distance, the second electronic device 20 or the third electronic device 30 can provide accurate voice recognition results.
[0089] Figure 2 is a diagram of a system according to another embodiment of the present disclosure.
[0090] Reference Figure 2 , the first electronic device 10, the second electronic device 20, the third electronic device 30, the first server 50 and the second server 60 correspond to the first electronic device 10, the second electronic device 20, the third electronic device 30, the first server 50 and the second server 60 of Figures 1A and 1B, respectively, and will not be further described.
[0091] Reference Figure 2 , when user 1 makes a voice 2, the first electronic device 10 can use a microphone provided in the first electronic device 10 to sense the voice 2 made by user 1 at a specific time (①-1). When the voice 2 is sensed, the first electronic device 10 can send a wake-up command to the second electronic device 20 and the third electronic device 30 to control the acquisition of voice data of the voice 2 (①-2, ①-3).
[0092] The first electronic device 10 can obtain the first voice data of the voice 2 (①-4), obtain the voice recognition result corresponding to the first voice data, and send the voice recognition result to the second electronic device 20 and the third electronic device 30 (⑥, ⑦).
[0093] At the same time, the second electronic device 20 receiving the wake-up command may obtain the second voice data (②) of voice 2, and the third electronic device 30 receiving the wake-up command may obtain the third voice data (③) of voice 2. For example, when a wake-up command is received during operation in a low power mode, the second electronic device 20 or the third electronic device 30 may operate in a normal mode, turn on the microphone, and obtain the second voice data and the third voice data of voice 2, respectively.
[0094] When the voice recognition corresponding to the first voice data is received from the first electronic device 10, the second electronic device 20 can use the second voice data of the voice 2 and the received voice recognition result as data for training to update the voice recognition model, and the third electronic device 30 can use the third voice data of the voice 2 and the voice recognition result received from the first electronic device 10 as data for training to update the voice recognition model. Operations ⑧ to ⑩ correspond to operations ⑧ to ⑩ in FIG. 1A, and repeated descriptions will be omitted.
[0095] Figure 3 is a diagram of a system according to another embodiment of the present disclosure.
[0096] Reference Figure 3 , Figure 3 The first electronic device 10, the second electronic device 20, the third electronic device 30, the first server 50 and the second server 60 correspond to the first electronic device 10, the second electronic device 20, the third electronic device 30, the first server 50 and the second server 60 of FIG. 1A and FIG. 1B, respectively, and repeated description will be omitted. At this time, the situation in which the first electronic device 10, the second electronic device 20 and the third electronic device 30 are in may be the same as that in FIG. 1A, FIG. 1B and Figure 2 different.
[0097] Reference Figure 3 , and Figure 1A, Figure 1B, and Figure 2 Differently, the first electronic device 10 and the second electronic device 20 may be at the same or similar distance from the user 1. However, the peripheral environments of the first electronic device 10 and the second electronic device 20 may be different from each other. For example, the first electronic device 10 may be located inside a room, and the second electronic device 20 may be located near a window. In this case, the ambient noise at the second electronic device 20 may be higher than the ambient noise at the first electronic device 10 due to noise from outside the window (e.g., traffic noise, etc.).
[0098] In this case, the environmental noise at the first electronic device 10 is relatively low, so the first electronic device 10 can obtain the first voice data of voice 2 (①), send the first voice data to the first server 50 (③), and obtain the voice recognition result corresponding to voice 2 (④). The first electronic device 10 can send the obtained voice recognition result to the second electronic device 20 where the environmental noise is relatively high (⑤).
[0099] The second electronic device 20 with relatively high environmental noise can obtain the second voice data of the voice 2 when the user 1 utters a voice (②), and obtain the voice recognition result corresponding to the voice 2 from the first electronic device 10 (⑤). The second electronic device 20 can send the obtained second voice data and the voice recognition result obtained from the first electronic device 10 as data for training to the second server (⑥), and control so that the second server 60 can use the received data for training to update the voice recognition model (⑦).
[0100] Therefore, even for the voice 2 received in an environment with high ambient noise, the second server 60 can use accurate voice recognition results to train the voice recognition model. When the user 1 utters a voice in the absence of the first electronic device 10, the second electronic device 20 located in the high ambient noise environment may be able to use an updated voice recognition model to provide a voice recognition result corresponding to the voice uttered by the user 1.
[0101] Figure 4 is a block diagram of a configuration of an electronic device according to an embodiment of the present disclosure.
[0102] Reference Figure 4 The electronic device 400 may be as shown in FIG. 1A, FIG. 1B, Figure 2 and Figure 3 Any one of the first electronic device 10 , the second electronic device 20 and the third electronic device 30 .
[0103] The electronic device 400 may include a processor 410, a microphone 420, a communicator 430, and a memory 440. However, the electronic device 400 may be composed of a processor 410, a microphone 420, a communicator 430, and a memory 440. Figure 4 For example, the electronic device 400 according to the embodiment may be included in at least one of a home appliance, a mobile computing device, and a server, or may be wirelessly or wiredly connected to at least one of the home appliance, the mobile computing device, and the server. Detailed examples of the processor 410, the microphone 420, the communicator 430, and the memory 440 may be respectively described in detail. Fig. 9 The processor 920, audio module 970, communication module 990 and memory 930 correspond to each other.
[0104] Microphone (or microphone) 420 may directly receive the voice of user 1 (or speaker). Microphone 420 may receive the voice of user 1 and provide voice data associated with the voice of user 1 to processor 410.
[0105] The processor 410 may execute a program stored in the memory 440 and control at least one other component (eg, hardware or software component) of the electronic device 400 connected to the processor 410 .
[0106] In one embodiment, the processor 410 may obtain first voice data of a voice uttered by user 1 at a specific time through the microphone 420. In addition, when the voice uttered by user 1 is detected, the processor 410 may send a wake-up command through the communicator 430 to control the second electronic device 20 to obtain second voice data about the voice.
[0107] The processor 410 may obtain a speech recognition result corresponding to the obtained first speech data. For example, the processor 410 may send the obtained first speech data to an external speech recognition server through the communicator 430. In response to the transmission of the first speech data, the processor 410 may send the speech recognition result to a second electronic device that has obtained the second speech data of the speech uttered by the user 1 at a specific time through the communicator 430, so that the second electronic device may use the speech recognition result as data for training a speech recognition model.
[0108] As an embodiment, the processor 410 may send time information related to the voice to the second electronic device. The time information related to the voice may include, for example, at least one of the time when the processor 410 recognizes the start of the voice or the time when the processor 410 recognizes the end of the voice.
[0109] Figure 5 is a diagram illustrating voices input to a first electronic device and a second electronic device according to an embodiment of the present disclosure.
[0110] Specifically, refer to Figure 5 , user 1 can start speaking voice 2 at time t0 and end speaking voice 2 at time t5. In this case, Figure 5 As shown, there may be a difference between the time when the voice signal reaches the first electronic device 10 located at a short distance from the user 1 and the time when the voice signal reaches the second electronic device 20 located at a long distance from the user 1 .
[0111] For example, the first electronic device 10 may recognize the beginning of the voice 2 of the user 1 at time t1, and may recognize the end of the voice 2 of the user 1 at time t6. The first electronic device 10 that recognizes the voice 2 of the user 1 at time t1 may send a wake-up command of the microphone of the second electronic device 20 to the second electronic device 20 that is far away from the user 1. The second electronic device 20 that receives the wake-up command may recognize the beginning of the voice 2 of the user 1 from time t2, and recognize the end of the voice 2 of the user 1 at time t7.
[0112] In this case, since the voice recognition result provided by the first electronic device 10 to the second electronic device 20 corresponds to the voice from time t1 to time t6, it may be different from the voice recognized from time t2 to time t7 by the second electronic device 20. Therefore, the first electronic device 10 may send time information related to the voice corresponding to the voice recognition result and time information about sending the wake-up command to the second electronic device 20 together with the voice recognition result.
[0113] The second electronic device 20 may use at least one of the speech recognition result, the time information related to the speech, and the time information of sending the wake-up command to correct the second speech data obtained by the second electronic device 20, wherein the second speech data will be used as data for training. For example, the second electronic device 20 may restore the missing second speech data according to the delay time of the speech due to the distance difference and the speech recognition after receiving the wake-up command to correspond to the speech recognition result.
[0114] As another example, the second electronic device 20 may extract only a portion of the voice recognition result received from the first electronic device 10 that corresponds to the second voice data obtained by the second electronic device 20. The extracted voice recognition result and the second voice data may be used as data for training. Specifically, the voice recognized by the first electronic device 10 may be "Annyeonghaseyo", and the voice recognized by the second electronic device 20 may be "Haseyo" recognized after receiving the wake-up command. In this case, the second electronic device 20 may extract only a portion of the voice recognition result received from the first electronic device 10 that corresponds to "Haseyo", and use the portion as data for training.
[0115] When the corrected second voice data is obtained, the second electronic device 20 may use the corrected second voice data and a voice recognition result corresponding to the first voice data obtained from the first electronic device 10 as data for training a voice recognition model.
[0116] In one embodiment, the processor 410 may obtain the second voice data about the voice uttered by the user 1 at a specific time through the microphone 420. For example, the processor 410 may receive a wake-up command for obtaining the second voice data from the first electronic device 10 through the communicator. In response to the wake-up command, the processor 410 may obtain the second voice data about the voice uttered by the user at a specific time through the microphone 420.
[0117] The processor 410 may receive, through the communicator 430 , a voice recognition result corresponding to the first voice data from the first electronic device that acquires the first voice data of the voice uttered by the user 1 at a specific time.
[0118] The processor 410 may use the second voice data and a voice recognition result corresponding to the first voice data as data for training a voice recognition model.
[0119] In addition, the processor 410 may receive time information related to the voice from the first electronic device 10 through the communicator 430. In this case, the processor 410 may correct the obtained second voice data by using the time information related to the voice. In addition, the processor 410 may use the corrected second voice data and the voice recognition result corresponding to the first voice data as data for training the voice recognition model.
[0120] In addition, when the speech recognition model is updated using the second speech data and the speech recognition result corresponding to the first speech data, the processor 410 may obtain third speech data associated with speech data newly uttered by the user at a time different from the specific time through the microphone 420. The processor 410 may obtain a speech recognition result corresponding to the third speech data by applying the obtained third speech data to the updated speech recognition model using the data for training.
[0121] In addition, when obtaining first voice data about a voice uttered by a user at a specific time, the processor 410 of the first electronic device 10 may transmit the obtained first voice data to the second electronic device 20 through the communicator 430. While obtaining third voice data of voices uttered by the user and another user, the processor 410 of the second electronic device 20 may obtain predicted voice data of the user's voice from the third voice data using the obtained first voice data, and use the predicted voice data and a voice recognition result corresponding to the first voice data as data for training a voice recognition model.
[0122] Figure 6 is a flow chart between a first electronic device and a second electronic device according to an embodiment of the present disclosure.
[0123] Reference Figure 6, in operation 601, the first electronic device 10 may sense a voice uttered by a user at a specific time. When the voice is sensed, in operation 602, the first electronic device 10 may send a wake-up command to the second electronic device 20 to control the second electronic device 20 to obtain second voice data of the voice. In addition, when the voice is sensed, in operation 603, the first electronic device 10 may obtain first voice data of the voice uttered by the user at a specific time through a microphone provided in the first electronic device 10. In operation 605, the first electronic device 10 may obtain a voice recognition result corresponding to the obtained first voice data. For example, the first electronic device 10 may apply the first voice data to a voice recognition model provided in the first electronic device 10 to obtain a voice recognition result. Alternatively, the first electronic device 10 may obtain a voice recognition result by applying the first voice data to a voice recognition model provided in an external server. In operation 606, the first electronic device 10 may send the obtained voice recognition result to the second electronic device 20.
[0124] In operation 604, the second electronic device 20 that has received the wake-up command may obtain second voice data of a voice uttered by a user at a specific time through a microphone provided in the second electronic device 20. In operation 607, the second electronic device 20 may update a voice recognition model used by the second electronic device 20 by using the obtained second voice data and a voice recognition result corresponding to the first voice data as data for training. For example, the second electronic device 20 may update the voice recognition model by applying the data for training to a voice recognition model provided in the second electronic device 20. Alternatively, the second electronic device 20 may update the voice recognition model by applying the data for training to a voice recognition model provided in an external server.
[0125] In operation 608, the second electronic device 20 may obtain third voice data regarding a voice uttered by the user at a specific time different from the specific time through a microphone provided in the second electronic device 20. In this case, in operation 609, the second electronic device 20 may obtain a voice recognition result corresponding to the third voice data by applying the obtained third voice data to the voice recognition model updated in operation 607.
[0126] Figure 7 is a flowchart of a first electronic device according to an embodiment of the present disclosure.
[0127] Reference Figure 7, the first electronic device 10 may be able to communicate with the second electronic device 20. In this case, the first electronic device 10 may exist in a situation where the user's voice is relatively well recognized compared to the second electronic device 20. For example, the first electronic device 10 may be an electronic device located at a first distance from the user, and the second electronic device 20 may be located at a second distance relatively farther than the first distance. Optionally, the ambient noise at the first electronic device 10 may be lower than the ambient noise at the second electronic device 20. Optionally, the performance of the microphone of the first electronic device 10 may be higher than the performance of the microphone of the second electronic device 20.
[0128] In this case, in operation 701, the first electronic device 10 may obtain first voice data about a voice uttered by a user at a specific time through a microphone provided in the first electronic device. In this case, when the voice uttered by the user is detected, the first electronic device 10 may send a wake-up command to the second electronic device 20 to control the second electronic device 20 to obtain second voice data.
[0129] In one embodiment, the first electronic device 10 may send the first voice data obtained by the first electronic device 10 to the second electronic device 20. The second electronic device 20 may obtain predicted voice data to be used as data for training by correcting the third voice data obtained by the second electronic device 20 using the received first voice data. For example, the third voice data may be third voice data of voices uttered by the user and other users. In this case, by using the first voice data, the second electronic device 20 may obtain predicted voice data of the user's voice expected to be obtained when the voice uttered by the user 1 is received through the microphone in the second electronic device 20 from the third voice data.
[0130] In operation 702, the first electronic device 10 may obtain a speech recognition result corresponding to the obtained first speech data. The speech recognition result may include, for example, at least one of text data, a user's intention, and a path rule.
[0131] In operation 703, the first electronic device 10 may transmit the speech recognition result to be used as data for training the speech recognition model to the second electronic device 20 that obtains the second speech data of the speech uttered by the user at a specific time. For example, the first electronic device 10 may transmit the speech recognition result used as data for learning to train at least one of the acoustic model and the language model included in the speech recognition model. At this time, the first electronic device 10 may also transmit the time information related to the speech together with the speech recognition result to the second electronic device 20.
[0132] Figure 8 is a flowchart of a second electronic device according to an embodiment of the present disclosure.
[0133] Reference Figure 8 , the second electronic device 20 may be able to communicate with the first electronic device 10. In this case, the second electronic device 20 may exist in a situation where it is relatively difficult to recognize the user's voice compared to the first electronic device 10. For example, the second electronic device 20 may be located at a second distance from the user, and the first electronic device 10 may be an electronic device 20 located at a first distance relatively closer to the user than the second distance. Optionally, the ambient noise at the second electronic device 20 may be higher than the ambient noise at the first electronic device 10. Optionally, the performance of the microphone of the second electronic device 20 may be lower than the performance of the microphone of the first electronic device 10.
[0134] In this case, in operation 801, the second electronic device 20 may obtain second voice data regarding a voice uttered by a user at a specific time through a microphone provided in the second electronic device 20. For example, the second electronic device 20 may receive a wake-up command for obtaining the second voice data from the first electronic device 10. In response to the wake-up command, the second electronic device 20 may obtain second voice data regarding a voice uttered by a user at a specific time through a microphone provided in the second electronic device 20.
[0135] In operation 802, the second electronic device 20 may obtain a voice recognition result corresponding to the first voice data from the first electronic device that obtains the first voice data of the voice uttered by the user at a specific time.
[0136] In operation 803, the second electronic device 20 may use the second voice data obtained in operation 801 and the voice recognition result corresponding to the first voice data obtained in operation 802 as data for training a voice recognition model.
[0137] As an embodiment, the second electronic device 20 may receive time information related to the voice and the voice recognition result from the first electronic device 10. In this case, the second electronic device 20 may use the time information related to the voice to correct the obtained second voice data. The second electronic device 20 may use the corrected second voice data and the voice recognition result corresponding to the first voice data obtained in operation 802 as data for training the voice recognition model.
[0138] In an embodiment, the second electronic device 20 may obtain third voice data about a newly uttered voice of the user at a time different from the specific time when the user uttered the voice in operation 801. In operation 803, the second electronic device 20 may apply the obtained third voice data to the voice recognition model updated using the data for training. As a result of the application, the second electronic device 20 may obtain a voice recognition result corresponding to the third voice data.
[0139] In an embodiment, the second electronic device 20 may obtain third voice data about the voice uttered by the user and another user. In this case, the second electronic device 20 may use the first voice data to obtain predicted voice data about the user's voice from the third voice data, wherein the predicted voice data is expected to be obtained when the second electronic device 20 receives the voice uttered by the user 1 through the microphone. The second electronic device 20 may use the obtained predicted voice data and the voice recognition result corresponding to the first voice data as data for training the voice recognition model.
[0140] Fig. 9 is a detailed block diagram of an electronic device in a network environment according to an embodiment of the present disclosure.
[0141] Reference Fig. 9 , the electronic device 901 may be FIG. 1A, FIG. 1B, Figure 2 and Figure 3 Any one of the first electronic device 10, the second electronic device 20 and the third electronic device 30 described in .
[0142] Reference Fig. 9 In the network environment 900, the electronic device 901 may communicate with the electronic device 902 through the first network 998 (e.g., a near-field range wireless communication network), or communicate with the electronic device 904 or the server 908 through the second network 999 (e.g., a long-range wireless communication network). According to an embodiment, the electronic device 901 may communicate with the electronic device 904 through the server 908. According to an embodiment, the electronic device 901 may include a processor 920, a memory 930, an input device 950, a sound output device 955, a display device 960, an audio module 970, and a communication module 990. In some embodiments, at least one of the components (e.g., the display device 960) may be omitted, or one or more other components may be added to the electronic device 901. In some embodiments, some of these components may be implemented in one integrated circuit.
[0143] The processor 920 can control at least one other component (e.g., hardware or software component) of the electronic device 901 connected to the processor 920 by executing software (e.g., program 940), and perform various data processing or operations. According to an embodiment, as at least part of the data processing or operation, the processor 920 may load a command or data received from another component (e.g., communication module 990) to the volatile memory 932, process the command or data stored in the volatile memory 932, and store the result data in the non-volatile memory 934. According to one embodiment, the processor 920 may include a main processor 921 (e.g., a central processing unit or an application processor) and an auxiliary processor 923 (e.g., a graphics processing unit, an image signal processor, a sensor hub processor, or a communication processor) that can operate together or independently. Additionally or optionally, the auxiliary processor 923 may use less power than the main processor 921, or may be set to be dedicated to a specified function. The auxiliary processor 923 may be implemented separately from the main processor 921, or as part of the main processor 921.
[0144] The auxiliary processor 923 may, for example, replace the main processor 921 when the main processor 921 is in an inactive state (e.g., sleep) or control a portion of a function or state related to at least one of the components of the electronic device 901 (e.g., display device 960, communication module 990) together with the main processor 921 when the main processor 921 is in an active state (e.g., execution of an application). According to one embodiment, the auxiliary processor 923 (e.g., an image signal processor or a communication processor) may be implemented as a part of other functionally related components (e.g., communication module 990). According to an embodiment, when a wake-up command is received from an external device, the auxiliary processor 923 may convert the main processor 921 to an active state.
[0145] The memory 930 may store various data used by at least one component (e.g., the processor 920) of the electronic device 901. The data may include, for example, software (e.g., a program 940) and input data or output data related to software instructions. The memory 930 may include a volatile memory 932 or a nonvolatile memory 934.
[0146] The program 940 may be stored as software in the memory 930 , and include, for example, an operating system 942 , middleware 944 , or an application 946 .
[0147] The input device 950 may receive a command or data intended for a component (e.g., processor 920) of the electronic device 901 from outside (e.g., a user) of the electronic device 901. The input device 950 may include, for example, a microphone, a mouse, a keyboard, or a digital pen (e.g., a stylus).
[0148] The sound output device 955 can output a sound signal to the outside of the electronic device 901. The sound output device 955 may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback, and the receiver may be used to receive an incoming call. According to one embodiment, the receiver may be implemented separately from the speaker, or as part of the speaker.
[0149] The display device 960 may visually provide information to the outside of the electronic device 901 (e.g., a user). The display device 960 may include, for example, a display, a holographic device, a projector, or a control circuit for controlling the device. According to an embodiment, the display device 960 may include a touch circuit configured to detect a touch or a sensor circuit (e.g., a pressure sensor) configured to measure the intensity of power generated by the touch.
[0150] The audio module 970 (e.g., a microphone) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module 970 can obtain sound through the input device 950, or output sound through the sound output device 955 or an external electronic device (e.g., electronic device 902) (e.g., a speaker or earphone) directly or wirelessly connected to the electronic device 901.
[0151] The connection terminal 978 may include a connector through which the electronic device 901 can be physically connected to an external electronic device (e.g., the electronic device 902). According to an embodiment, the connection terminal 978 may include, for example, a high-definition multimedia interface (HDMI) connector, a universal serial bus (USB) connector, a secure digital (SD) card connector, or an audio connector (e.g., a headphone connector).
[0152] The communication module 990 may support establishing a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device 901 and an external electronic device (e.g., electronic device 902, electronic device 904, or server 908), and communicating through the established communication channel. The communication module 990 may include one or more communication processors that operate independently of the processor 920 (e.g., an application processor) and support direct (e.g., wired) communication or wireless communication. According to an embodiment, the communication module 990 may include a wireless communication module 992 (e.g., a cellular communication module, a near-field wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 994 (e.g., a local area network (LAN) communication module or a power line communication module). The corresponding communication module in these communication modules may communicate with an external electronic device via a first network 998 (e.g., Bluetooth, Wi-Fi Direct, or a near-field communication network, such as an infrared data association (IrDA)) or a second network 999 (e.g., a telecommunications network, such as a cellular network, the Internet, or a computer network (e.g., LAN or WAN)). These types of communication modules may be incorporated into one component (eg, a single chip) or implemented with multiple components (eg, multiple chips) that are separate from each other.
[0153] At least part of the components may be interconnected between peripheral devices through a communication method such as a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI) and exchange signals (e.g., commands or data) with each other.
[0154] According to one embodiment, a command or data may be sent or received between the electronic device 901 and the electronic device 904 via the server 908 connected to the second network 999. Each of the electronic devices 902 and 904 may be a device of the same or different type as the electronic device 901. According to an embodiment, all or part of the operations performed by the electronic device 901 may be performed by one or more external devices in the electronic device 902, the electronic device 904, or the server 908. For example, when the electronic device 901 must automatically perform a function or service, or in response to a request from a user or another device, the electronic device 901 may request one or more external electronic devices to perform at least a part of the function or service, instead of performing the function or service alone, or request one or more external electronic devices to perform at least a part of the function or service in addition to performing the function or service alone. The one or more external electronic devices that have received the request may perform at least a part of the requested function or service, or an additional function or service associated with the request, and send the execution result to the electronic device 901. The electronic device 901 may process the result as is or otherwise, and provide the result as at least a part of a response to the request.For this purpose, for example, cloud computing, distributed computing or client-server computing technology may be used.
[0155] Fig.10 is a diagram illustrating a system in the presence of a plurality of speakers according to an embodiment of the present disclosure.
[0156] Reference Fig.10 , Fig.10 The first electronic device 10, the second electronic device 20, the third electronic device 30, the first server 50 and the second server 60 correspond to the first electronic device 10, the second electronic device 20, the third electronic device 30, the first server 50 and the second server 60 of Figures 1A and 1B, respectively, and redundant descriptions will be omitted.
[0157] Reference Fig.10 , showing a first user 1, a second user 4, and a third user 7. In this case, the first user 1 may be arranged closer to the first electronic device 10, the second user 4 may be arranged closer to the second electronic device 20, and the third user may be arranged closer to the third electronic device 30.
[0158] In this case, the first user 1, the second user 4, and the third user 7 may utter a voice together at a specific time (e.g., at a specific time or during a specific time). For example, at a specific time, the user 1 may utter a voice 2 "What time is it now?", the second user 4 may utter a voice 5 "The weather is good", and the third user 7 may utter a voice 8 "I'm hungry".
[0159] In this case, the microphone of each of the first electronic device 10, the second electronic device 20, and the third electronic device 30 may receive a voice signal emitted by each of the first user 1, the second user 4, and the third user 7, or receive a voice signal emitted by another user together. In the embodiments of FIGS. 11A and 11B , it is assumed that the microphone of the first electronic device 10 receives a voice signal emitted by the first user 1, the microphone of the second electronic device 20 receives a voice signal emitted by the first user 1 and the second user 4, and the microphone of the third electronic device 30 receives a voice signal emitted by the first user 1 and the third user 7.
[0160] In the above case, the first electronic device 10 located at a first distance (e.g., a short distance) from the first user 1 can obtain the first voice data of the voice 2 emitted by the first user 1 at a specific time through the microphone set in the first electronic device 10 (①). At this time, the second electronic device 20 located at a second distance (e.g., an intermediate distance) from the first user 1 can obtain the second voice data, wherein the second voice data includes not only the voice 2 emitted by the first user 1 at a specific time through the microphone set in the second electronic device 20, but also the voice 5 emitted by the second user 4 located closer to the second electronic device 20 at a specific time (②). In addition, the third electronic device 30 located at a third distance (e.g., a long distance) from the user 1 can obtain the third voice data, wherein the third voice data includes not only the voice 2 emitted by the first user 1 at a specific time through the microphone set in the third electronic device 30, but also the voice 8 emitted by the third user 7 located closer to the third electronic device 30 at a specific time (③).
[0161] exist Fig.10 In the embodiment of the present invention, when the first electronic device 10 obtains the first voice data, the first electronic device 10 may obtain a voice recognition result corresponding to the obtained first voice data. For example, the first electronic device 10 may apply the obtained first voice data to a voice recognition model provided in the first electronic device and obtain a voice recognition result. Optionally, the first electronic device 10 may send the first voice data to a first server 50 (④) that can communicate with the first electronic device 10. The first electronic device 10 may obtain a voice recognition result corresponding to the first voice data from the first server 50 (⑤). At this time, the voice recognition result corresponding to the first voice data may represent a voice recognition result corresponding to the voice uttered by the first user. Various methods for the first electronic device 10 to obtain a voice recognition result from the first server 50 have been described in FIG. 1A, and redundant descriptions will be omitted.
[0162] When the voice recognition result is obtained, the first electronic device 10 may send the voice recognition result corresponding to the voice 2 emitted by the first user 1 and the first voice data about the voice 2 emitted by the first user 1 to the second electronic device 20 (⑥). In addition, the first electronic device 10 may send the voice recognition result corresponding to the voice 2 emitted by the first user 1 and the first voice data about the voice 2 emitted by the first user to the third electronic device 30 (⑦). In an embodiment, each of the voice recognition result corresponding to the voice 2 emitted by the first user 1 and the first voice data of the voice 2 emitted by the first user 1 may be sent to the second electronic device 20 or the third electronic device 30, respectively, with a time difference. For example, after sending the voice recognition result corresponding to the voice 2 emitted by the first user 1, the first voice data about the voice 2 emitted by the first user 1 may be sent, and after sending the first voice data of the voice 2 emitted by the first user 1, the voice recognition result corresponding to the voice 2 emitted by the first user 1 may be sent.
[0163] The second electronic device 20 may use the speech recognition result corresponding to the speech 2 emitted by the first user 1 as a part of the data for training. For this purpose, the second electronic device 20 may extract the predicted speech data of the speech of the first user 1 from the second speech data by using the first speech data of the speech 2 emitted by the first user 1 emitted by the first electronic device 10 in operation ⑥, wherein the speech 2 emitted by the first user 1 and the speech 5 emitted by the second user 4 obtained by the second electronic device 20 in operation ② are combined in the second speech data. The predicted speech data may be the speech data of the speech 2 emitted by the first user 1 in the second speech data obtained by the microphone of the second electronic device 20.
[0164] Specifically, the first voice data of the voice 2 emitted by the first user 1 obtained by the first electronic device 10 can be represented as A, and the second voice data (in which the voices 5 emitted by the first user 1 and the second user 4 are combined) obtained by the second electronic device 20 can be represented as A'+B'.
[0165] At this time, the second voice data-first voice data=(A'+B')-A=B'+residual A. Here, the second voice data-(second voice data-first voice data)=(A'+B')-(B'+residual A)=A'-residual A. Therefore, A', which is the voice data of the voice uttered by the first user 1 obtained by the second electronic device 20, can be the value of the residual A added by the second electronic device 20 to the first voice data obtained from the first electronic device 10.
[0166] The residual A may be a value of a speech environment difference between the first electronic device 10 and the second electronic device 20. For example, the residual A may be a difference between first voice data A of the first user 1 obtained by the first electronic device 10 and second voice data A' of the first user 1 obtained by the second electronic device 2 when only the first user 1 is speaking. Optionally, in the non-speech part, the residual A may be a difference between data of an ambient signal obtained by a microphone provided in the first electronic device 10 and data of an ambient signal obtained by a microphone provided in the second electronic device 20.
[0167] Through the above-mentioned processing, when the predicted voice data A' is obtained, the second electronic device 20 can form the predicted voice data and the voice recognition result corresponding to the first voice data received in operation ⑥ into a set, and use it as data for training. For example, when the voice recognition model is located in the second electronic device 20, the second electronic device 20 can apply {the predicted voice data generated in the second electronic device 20 and the voice recognition result obtained by the first electronic device 10} to the voice recognition model as data for training, and update the voice recognition model located in the second electronic device 20 (⑧).
[0168] In addition, the third electronic device 30 may use a speech recognition result corresponding to the speech 2 uttered by the first user 1 as a part of the data for training. To this end, the third electronic device 30 may extract predicted speech data of the speech of the first user 1 from the third speech data by using the first speech data of the speech 2 uttered by the first user 1 obtained by the first electronic device 10 in operation ⑦, wherein the speech 8 uttered by the first user 1 and the third user 7 obtained by the third electronic device 30 in operation ③ are combined in the third speech data. The predicted speech data may be speech data of the speech 2 uttered by the first user 1 in the third speech data obtained by the microphone of the third electronic device 30. The method for extracting the predicted speech data has been described with respect to the second electronic device 20 and will not be described further.
[0169] The third electronic device 30 may form a set of the predicted voice data and the voice recognition result corresponding to the first voice data received in operation ⑦, and use the set as data for training. For example, when the voice recognition model is located at the second server 60, the third electronic device 30 may send {the predicted voice data generated by the third electronic device 30, the voice recognition result obtained by the first electronic device 10} as data for training to the second server 60 where the voice recognition model is located (⑨). The second server 60 may apply the received data for training to the voice recognition model, and update the voice recognition model stored in the second server 60 (⑩).
[0170] 11A and 11B are diagrams of a system that considers a discourse environment according to an embodiment of the present disclosure.
[0171] The first electronic device 10, the second electronic device 20, the third electronic device 30, the first server 50 and the second server 60 of Figure 11A correspond to the first electronic device 10, the second electronic device 20, the third electronic device 30, the first server 50 and the second server 60 of Figures 1A and 1B, respectively, and redundant descriptions will be omitted.
[0172] 11A , when user 1 utters voice 2 , the first electronic device 10 may obtain first voice data of voice 2 uttered by user 1 at a specific time through a microphone provided in the first electronic device 10 (①).
[0173] At this time, the second electronic device 20 located at a second distance (for example, an intermediate distance) from the user 1 can obtain the second voice data of the voice 2 uttered by the user 1 at a specific time through the microphone set in the second electronic device 20 (②).
[0174] The second electronic device 20 may transmit the second voice data of the voice 2 uttered by the user 1 to the first electronic device 10 (②-1).
[0175] Using the second voice data obtained from the second electronic device 20, the first electronic device 10 may obtain a first speech environment difference between the first electronic device 10 and the second electronic device 20, and store the obtained first speech environment difference (②-2).
[0176] Specifically, the first voice data of the voice 2 emitted by the user 1 obtained by the first electronic device 10 can be represented as A, and the second voice data of the voice 2 emitted by the user 1 obtained by the second electronic device 20 can be represented as A'. At this time, the second voice data-first voice data=A-A'=residual A. The residual A can be stored as the first speech environment difference between the first electronic device 10 and the second electronic device 20.
[0177] In addition, the third electronic device 30 located at a third distance (eg, long distance) from the user 1 can also obtain third voice data (③) of the voice 2 uttered by the user 1 at a specific time through a microphone provided in the third electronic device 30 .
[0178] The third electronic device 30 may transmit the third voice data of the voice 2 uttered by the user 1 to the first electronic device 10 (③-1).
[0179] Using the third voice data obtained from the third electronic device 30, the first electronic device 10 may obtain a second speech environment difference between the first electronic device 10 and the third electronic device 30, and store the obtained second speech environment difference (③-2).
[0180] 11B , user 1 may utter voice 9 at a time different from the specific time at which voice 2 is uttered. The first electronic device 10 may obtain fourth voice data of voice 9 uttered by user 1 through the provided microphone (④). The first electronic device 10 may obtain a voice recognition result corresponding to the obtained fourth voice data. For example, the first electronic device 10 may obtain a voice recognition result by applying the voice recognition result corresponding to the obtained fourth voice data to a voice recognition model provided in the first electronic device 10. Optionally, the first electronic device 10 may send the fourth voice data to a first server 50 capable of communicating with the first electronic device 10 (⑤). The first electronic device 10 may obtain a voice recognition result corresponding to the fourth voice data from the first server 50 (⑥).
[0181] At the same time, when the user receives the voice 9 uttered by the user 1 through the microphone in the second electronic device 20, the first electronic device 10 can generate the expected first predicted voice data by using the obtained fourth voice data and the first speech environment difference value stored in operation (②-2). For example, the first electronic device 10 can generate the first predicted voice data by adding the obtained fourth voice data to the first speech environment difference value. The first electronic device 10 can send the voice recognition result corresponding to the first predicted voice data and the fourth voice data obtained in operation ⑥ to the second electronic device 20 (⑦). The second electronic device 20 can update the voice recognition model (⑧) by applying {the first predicted voice data generated in the first electronic device 10 and the voice recognition result obtained from the first electronic device 10} as data for training to the voice recognition model.
[0182] In addition, by using the obtained fourth voice data and the second speech environment difference value stored in operation (③-2), the first electronic device 10 can generate the second predicted voice data expected to be obtained when the voice 9 uttered by the user 1 is received via the microphone. For example, the first electronic device 10 can generate the second predicted voice data by adding the obtained fourth voice data to the second voice environment difference value. The first electronic device 10 can send the voice recognition result corresponding to the second predicted voice data and the fourth voice data obtained in operation ⑥ to the third electronic device 30 (⑨). The third electronic device 30 can form {the second predicted voice data generated by the first electronic device 10, the voice recognition result obtained from the first electronic device 10} into a set and use the data as data for training. For example, when the voice recognition model is located in the second server 60, the third electronic device 30 can send {the second predicted voice data generated in the first electronic device 10, the voice recognition result obtained from the first electronic device 10} as data for training to the second server 60 where the voice recognition model is located (⑩). By applying the received data for training to the voice recognition model, the second server 60 can update the voice recognition model stored in the second server 60 .
[0183] In one embodiment, the first electronic device 10 may send {the second predicted speech data generated in the first electronic device 10, the speech recognition result obtained by the first electronic device 10} directly to the second server 60 without passing through the third electronic device 30. The second server 60 may apply the received data for training to the speech recognition model and update the speech recognition model stored in the second server 60.
[0184] In one embodiment, the above-mentioned speech environment difference value can be measured even when the user 1 does not utter a voice. For example, after purchasing the first electronic device 10, the second electronic device 20, and the third electronic device 30, the user can activate the microphone of the electronic device for a predetermined time. In this case, based on the difference value of the data in the environmental signal received by the microphone of the above-mentioned electronic device, the speech environment difference value for generating the predicted speech data can be obtained.
[0185] The term "module" used in the present disclosure may include a unit composed of hardware, software or firmware, and may be used interchangeably with terms such as, for example, logic, logic block, component or circuit. A "module" may be a component or the smallest unit or part of an overall structure that performs one or more functions. For example, according to an embodiment, a module may be configured as an application specific integrated circuit (ASIC).
[0186] Meanwhile, various embodiments of the present disclosure may be implemented in software (e.g., program 940), which includes instructions stored on a machine-readable storage medium (e.g., memory 440, internal memory 936, or external memory 938) readable by a machine (e.g., first electronic device 10, second electronic device 20, third electronic device 30, electronic device 901). For example, a processor (e.g., processor 410, processor 920) of a device (e.g., first electronic device 10, second electronic device 20, third electronic device 30) may call at least one instruction from one or more instructions stored in a storage medium and execute the called instruction. This enables the device to be operated to execute at least one instruction according to the at least one called instruction. One or more instructions may include code generated by a compiler or code executable by an interpreter. A machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory" only means that the storage medium does not include a signal (e.g., an electromagnetic wave) but is tangible, and the case where data is not stored semi-permanently in the storage medium is distinguished from the case where data is temporarily stored in the storage medium.
[0187] According to an embodiment, the method according to various embodiments disclosed herein may be provided in a computer program product. The computer program product may be exchanged between a seller and a buyer as a commodity. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)), or through an application store (e.g., PlayStore). TM ) is distributed online directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily or at least temporarily stored in a storage medium such as a memory of a manufacturer's server, an application store's server, or a relay server.
[0188] Each element (e.g., module or program) according to various embodiments may be composed of a single entity or multiple entities, and some sub-elements in the above-mentioned sub-elements may be omitted. These elements may be further included in various embodiments. Alternatively or additionally, some elements (e.g., modules or programs) may be integrated into one entity to perform the same or similar functions performed by each corresponding element before integration. According to various embodiments, the operations performed by modules, programs or other elements may be performed sequentially, in parallel, repeatedly or heuristically, or at least some operations may be performed in different orders.
[0189] While the present disclosure has been shown and described with reference to various embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents.
Claims
1. A method for providing data for training by a first electronic device, the method include: obtaining first voice data of a voice uttered by a user at a specific time through a microphone of the first electronic device; obtaining a first speech recognition result including a first text by inputting the first speech data into a first speech recognition model, wherein the first text corresponds to the first speech data; and sending the first voice recognition result to a second electronic device that obtains second voice data of the voice uttered by the user at the specific time, The first speech recognition result and the second speech data are used to train a second speech recognition model to obtain a second speech recognition result including a second text, wherein the second text corresponds to the second speech data, The performance of the microphone of the second electronic device is lower than the performance of the microphone of the first electronic device.
2. The method for providing data for training by a first electronic device according to claim 1, wherein include: Based on sensing the voice uttered by the user, a wake-up command is sent to the second electronic device to control the second electronic device to obtain second voice data of the voice.
3. The method for providing data for training by a first electronic device according to claim 1, further comprising: include: The obtained first voice data is transmitted to the second electronic device.
4. The method for providing data for training by a first electronic device according to claim 1, in, The first electronic device is positioned at a first distance from the user, and the second electronic device is positioned at a second distance, wherein the second distance is a greater distance than the first distance.
5. The method for providing data for training by a first electronic device according to claim 1, in, The ambient noise at the second electronic device is higher than the ambient noise at the first electronic device.
6. A method for obtaining data for training by a second electronic device, the method include: obtaining second voice data of a voice uttered by a user at a specific time through a microphone of the second electronic device; Receiving a first speech recognition result including a first text from a first electronic device, wherein the first electronic device obtains first speech data of a speech uttered by a user at the specific time, the first text corresponds to the first speech data, and the first speech recognition result is obtained by using a first speech recognition model; and training a second speech recognition model using the second speech data and the first speech recognition result corresponding to the first speech data to obtain a second speech recognition result including a second text, wherein the second text corresponds to the second speech data, The performance of the microphone of the second electronic device is lower than the performance of the microphone of the first electronic device.
7. The method for obtaining data for training by a second electronic device according to claim 6, further comprising: include: receiving a wake-up command from the first electronic device to obtain second voice data, The step of obtaining the second voice data includes: in response to the wake-up command, obtaining the second voice data of the voice uttered by the user at the specific time through the microphone of the second electronic device.
8. The method for obtaining data for training by a second electronic device according to claim 6, further comprising: include: receiving time information associated with the speech from the first electronic device; as well as correcting the obtained second speech data using time information associated with the speech, Among them, the step of using the second voice data and the first voice recognition result corresponding to the first voice data to train the second voice recognition model includes: using the corrected second voice data and the first voice recognition result corresponding to the first voice data as data for training the second voice recognition model.
9. The method for obtaining data for training by a second electronic device according to claim 6, further comprising: include: obtaining third voice data of a voice uttered by the user at a time different from the specific time; applying the obtained third speech data to a second speech recognition model updated using data used to train the second speech recognition model; as well as Based on applying the obtained third voice data to the second voice recognition model updated using the data for training the second voice recognition model, a first voice recognition result corresponding to the third voice data is obtained.
10. The method for obtaining data for training by a second electronic device according to claim 6, further comprising: include: obtaining third voice data of voices uttered by the user and another user; obtaining first voice data from a first electronic device; Using the first voice data, obtaining predicted voice data of the voice uttered by the user from the third voice data; as well as The predicted speech data and a first speech recognition result corresponding to the first speech data are used as additional data for training a second speech recognition model.
11. A first electronic device, include: A microphone configured to receive speech from a user; a communicator configured to communicate with a second electronic device; at least one processor configured to execute at least one instruction; as well as a memory configured to store the at least one instruction, The at least one processor is configured to: Acquire first voice data of a voice uttered by a user at a specific time through a microphone, obtaining a first speech recognition result including a first text by inputting the first speech data into a first speech recognition model, wherein the first text corresponds to the first speech data, and sending the first voice recognition result to a second electronic device that obtains second voice data of the voice uttered by the user at the specific time, The first speech recognition result and the second speech data are used to train a second speech recognition model to obtain a second speech recognition result including a second text, wherein the second text corresponds to the second speech data, The performance of the microphone of the second electronic device is lower than the performance of the microphone of the first electronic device.
12. The first electronic device according to claim 11, in, The at least one processor is configured to, based on the at least one instruction being executed, send a wake-up command to the second electronic device based on sensing a voice uttered by a user, so as to control the second electronic device to obtain second voice data of the voice.
13. The first electronic device according to claim 11, in, The at least one processor is configured to: transmit the obtained first voice data to the second electronic device based on the execution of the at least one instruction.
14. The first electronic device according to claim 11, in, The first electronic device is positioned at a first distance from the user, and the second electronic device is positioned at a second distance, wherein the second distance is a greater distance than the first distance.
15. A second electronic device, include: A microphone configured to receive speech from a user; a communicator configured to communicate with a first electronic device; at least one processor configured to execute at least one instruction; as well as a memory configured to store the at least one instruction, The at least one processor is configured to: Acquire second voice data of the voice uttered by the user at a specific time through a microphone, receiving, through the communicator, a first speech recognition result including a first text from a first electronic device that obtains first speech data of a speech uttered by a user at the specific time, wherein the first text corresponds to the first speech data and the first speech recognition result is obtained by using a first speech recognition model; and training a second speech recognition model using the second speech data and the first speech recognition result corresponding to the first speech data to obtain a second speech recognition result including a second text, wherein the second text corresponds to the second speech data, The performance of the microphone of the second electronic device is lower than the performance of the microphone of the first electronic device.
