Electronic device and control method thereof

Abstract

This electronic device comprises a memory for storing instructions and at least one processor, wherein the instructions, when collectively or individually executed by the at least one processor, cause the electronic device to: acquire a first token corresponding to input text of a first language; update a window corresponding to the first language in which the first token is stored, on the basis of the length of the first token; acquire a second token of a second language via an artificial intelligence model on the basis of a plurality of tokens stored in the updated window corresponding to the first language; and when the acquired second token of the second language includes at least one token corresponding to a plurality of tokens pre-stored in a window corresponding to the second language, update the window corresponding to the second language on the basis of the at least one token.

Description

Electronic device and control method thereof

[0001] The present disclosure relates to an invention concerning an electronic device and a method for controlling the same, and more specifically, includes an electronic device and a method for controlling the same for improving translation accuracy.

[0002] Recently, thanks to advancements in electronic technology, users can receive translation services through electronic devices.

[0003] However, conventionally, there was a problem of translation delay because translation began only after identifying the end of the sentence.

[0004] Accordingly, there is a recent demand to explore methods for translating source language text in real time.

[0005] An electronic device according to one embodiment of the present disclosure comprises: a memory for storing instructions; and at least one processor; wherein, when the instructions are executed collectively or individually by the at least one processor, the electronic device may acquire a first token corresponding to text of a first language input, update a window corresponding to the first language in which the first token is stored based on the length of the first token, acquire a second token of a second language through an artificial intelligence model based on a plurality of tokens stored in the updated window corresponding to the first language, and if there is at least one token included in the acquired second token of the second language corresponding to a plurality of tokens already stored in the window corresponding to the second language, update the window corresponding to the second language based on the at least one token.

[0006] When the above instructions are executed collectively or individually by the at least one processor, the electronic device may store the acquired first token in a first buffer and update a window corresponding to the first language, which identifies the number of stored tokens according to the first language, based on the number of tokens included in the first token.

[0007] The first token may be one of a plurality of tokens of the first language. When the instructions are executed collectively or individually by the at least one processor, the electronic device may update the window corresponding to the first language by storing some tokens included in the first token in the window corresponding to the first language if the length of the first token is longer than the length of the window corresponding to the first language.

[0008] The first token may be one of a plurality of tokens of the first language. When the instructions are executed collectively or individually by the at least one processor, the electronic device may update the window corresponding to the first language by removing the preceding token among the plurality of tokens of the first language and storing the added token among the plurality of tokens stored in the window corresponding to the first language.

[0009] The second token may be one of a plurality of tokens of the second language. When the instructions are executed collectively or individually by the at least one processor, the electronic device may store the second token of the second language in a second buffer and identify whether the plurality of tokens of the second language correspond to a plurality of tokens stored in a window corresponding to the second language more than a preset number of times.

[0010] When the above instructions are executed collectively or individually by the at least one processor, the electronic device may store at least one token included in a plurality of tokens of the second language in a third buffer.

[0011] The second token may be one of a plurality of tokens of the second language. When the instructions are executed collectively or individually by the at least one processor, the electronic device may update by removing a token with a higher priority among the plurality of tokens of the second language if the length of the second token of the second language is longer than the window length corresponding to the second language.

[0012] The second token may be one of a plurality of tokens of the second language. A window corresponding to the second language may store a plurality of tokens stored in the second buffer and a third buffer in which at least one of the plurality of tokens of the second language is stored.

[0013] When the above instructions are executed collectively or individually by the at least one processor, the electronic device may set the length of a window corresponding to the first language based on the first language, and set the length of a window corresponding to the second language based on the second language.

[0014] When the above instructions are executed collectively or individually by the at least one processor, if the electronic device identifies that there is at least one token included in the acquired second token of the second language corresponding to a plurality of tokens stored in the window corresponding to the second language, the device may determine the at least one token and update the window corresponding to the second language including the determined at least one token.

[0015] A method for controlling an electronic device according to one embodiment of the present disclosure may include: a step of acquiring a first token corresponding to text of a first language input; a step of updating a window corresponding to the first language in which the first token is stored based on the length of the first token; a step of acquiring a second token of a second language through an artificial intelligence model based on a plurality of tokens stored in the updated window corresponding to the first language; and a step of updating the window corresponding to the second language based on at least one token if there is at least one token included in the acquired second token of the second language corresponding to a plurality of tokens stored in the window corresponding to the second language.

[0016] The method may include the step of storing the first token obtained above in a first buffer; and the step of updating a window corresponding to the first language, which identifies the number of stored tokens according to the first language, based on the number of tokens included in the first token.

[0017] If the length of the first token is longer than the length of the window corresponding to the first language, the method may include the step of storing some tokens included in the first token in the window corresponding to the first language to update the window corresponding to the first language.

[0018] The first token may be one of a plurality of tokens of the first language. The method may include the step of removing a front token among the plurality of tokens of the first language; and the step of updating the window corresponding to the first language by storing an added token among the plurality of tokens stored in the window corresponding to the first language.

[0019] The second token may be one of a plurality of tokens of the second language. The method may include the step of storing the second token of the second language in a second buffer; and the step of identifying whether the plurality of tokens of the second language correspond to a plurality of tokens stored in a window corresponding to the second language more than a preset number of times.

[0020] It may include the step of storing at least one token included in the second token of the second language obtained above in a third buffer.

[0021] If the length of the second token of the second language is longer than the window length corresponding to the second language, the method may include the step of removing the token with the highest priority among the plurality of tokens included in the second token to update it.

[0022] The window corresponding to the second language may include the step of storing a plurality of tokens stored in the second buffer and a third buffer in which at least one token included in the second token of the acquired second language is stored.

[0023] It may include the step of setting the length of a window corresponding to the first language based on the first language; and the step of setting the length of a window corresponding to the second language based on the second language.

[0024] A non-transient computer-readable recording medium comprising a program for executing a method of controlling an electronic device according to one embodiment of the present disclosure may include: a step of acquiring a first token corresponding to text of a first language input; a step of updating a window corresponding to the first language in which the first token is stored based on the length of the first token; a step of acquiring a second token of a second language through an artificial intelligence model based on a plurality of tokens stored in the updated window corresponding to the first language; and a step of updating the window corresponding to the second language based on at least one token if there is at least one token included in the acquired second token of the second language corresponding to a plurality of tokens already stored in the window corresponding to the second language.

[0025] FIG. 1 is a drawing for explaining an embodiment of an electronic device according to one embodiment of the present disclosure, and

[0026] FIG. 2 is a block diagram for explaining an electronic device according to one embodiment of the present disclosure, and

[0027] FIGS. 3 to 8 are drawings for explaining a method of updating windows corresponding to a source language and a second language and translating in real time according to an embodiment of the present disclosure, and

[0028] FIG. 9 is a flowchart illustrating a method for controlling an electronic device according to one embodiment of the present disclosure.

[0029] The embodiments described herein are subject to various modifications and may have various forms; specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope of specific embodiments and should be understood to include various modifications, equivalents, and / or alternatives of the embodiments of the present disclosure. In relation to the description of the drawings, similar reference numerals may be used for similar components.

[0030] In describing the present disclosure, if it is determined that a detailed description of related known functions or configurations could unnecessarily obscure the essence of the present disclosure, such detailed description is omitted.

[0031] Additionally, the following embodiments may be modified in various other forms, and the scope of the technical concept of the present disclosure is not limited to the following embodiments. Rather, these embodiments are provided to make the present disclosure more faithful and complete and to fully convey the technical concept of the present disclosure to those skilled in the art.

[0032] The terms used in this disclosure are used merely to describe specific embodiments and are not intended to limit the scope of the rights. The singular expression includes the plural expression unless the context clearly indicates otherwise.

[0033] In the present disclosure, expressions such as “have,” “may have,” “include,” or “may include” indicate the presence of such features (e.g., numerical values, functions, actions, or components such as parts) and do not exclude the presence of additional features.

[0034] In the present disclosure, expressions such as “A or B,” “at least one of A or / and B,” or “one or more of A or / and B” may include all possible combinations of items listed together. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” may refer to cases including (1) at least one A, (2) at least one B, or (3) both at least one A and at least one B.

[0035] Expressions such as "first," "second," "first," or "second" used in this disclosure may modify various components regardless of order and / or importance, and are used only to distinguish one component from another and do not limit said components.

[0036] Where it is stated that a certain component (e.g., a first component) is "(operatively or communicatively) coupled with / to" or "connected to" another component (e.g., a second component), it should be understood that the said certain component may be directly connected to the said other component or connected through another component (e.g., a third component).

[0037] On the other hand, when it is stated that a certain component (e.g., a first component) is "directly connected" or "directly coupled" to another component (e.g., a second component), it may be understood that no other component (e.g., a third component) exists between said certain component and said other component.

[0038] As used in this disclosure, the expression “configured to” may be replaced, depending on the context, with, for example, “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured to” may not necessarily mean only “specifically designed to” in hardware.

[0039] Instead, in some situations, the expression “device configured to do something” may mean that the device is “capable of doing something” together with other devices or components. For example, the phrase “processor configured (or set) to perform A, B, and C” may mean a dedicated processor for performing those operations (e.g., an embedded processor), or a generic-purpose processor (e.g., a CPU or application processor) capable of performing those operations by executing one or more software programs stored in a memory device.

[0040] In the embodiments, a 'module' or 'part' performs at least one function or operation and may be implemented in hardware or software, or a combination of hardware and software. Additionally, a plurality of 'modules' or a plurality of 'parts' may be integrated into at least one module and implemented by at least one processor, except for the 'module' or 'part' that needs to be implemented in specific hardware.

[0041] Meanwhile, the various elements and areas in the drawings are depicted schematically. Accordingly, the technical concept of the present invention is not limited by the relative sizes or spacing depicted in the attached drawings.

[0042] Hereinafter, various embodiments of the present invention will be described in detail using the attached drawings.

[0043] FIG. 1 is a drawing for explaining an embodiment of an electronic device according to one embodiment of the present disclosure. The electronic device (100) may be implemented as a TV as shown in FIG. 1, but this is only one embodiment, and it is obvious that it may be implemented as various types of electronic devices such as a tablet, laptop computer, smartphone, camera, PC, etc.

[0044] Meanwhile, for the sake of convenience of explanation, the following description is based on the premise that the electronic device (100) includes a display (not shown), but this is merely one embodiment, and it is obvious that the electronic device (100) can output video data or audio signals using HDMI or DP (Display Port) without including a display.

[0045] If the electronic device (100) does not include a display (not shown), the electronic device (100) may transmit data to an external device that includes a display and output video data or audio signals from the external device.

[0046] First, the electronic device (100) can receive video data. At this time, the video data may include scene data, subtitle data (or text data), and audio data.

[0047] There may be an embodiment in which an electronic device (100) performs an action corresponding to a user voice signal received through a microphone (not shown).

[0048] According to one embodiment, the electronic device (100) can control the display (130) based on a user voice signal received through a microphone (not shown). For example, when a user voice signal for displaying content A is received, the electronic device (100) can control the display (130) to display content A.

[0049] According to one embodiment, the electronic device (100) can control an external display device connected to the electronic device (100) based on a user voice signal received through a microphone (not shown). The electronic device (100) can generate a control signal to control the external display device so that an operation corresponding to the user voice signal is performed on the external display device, and can transmit the generated control signal to the external display device. The electronic device (100) can store a remote control application for controlling the external display device. Furthermore, the electronic device (100) can transmit the generated control signal to the external display device using at least one communication method among Bluetooth, Wi-Fi, or infrared. For example, when a user voice signal for displaying content A is received, the electronic device (100) can transmit a control signal to the external display device to control the display of content A on the external display device. The electronic device (100) may refer to various terminal devices capable of installing a remote control application, such as a smartphone or an AI speaker.

[0050] According to one embodiment, the electronic device (100) may use a remote control device to control an external display device connected to the electronic device (100) based on a user voice signal received through a microphone (not shown). The electronic device (100) may transmit a control signal to the remote control device to control the external display device so that an operation corresponding to the user voice signal is performed on the external display device. The remote control device may transmit the control signal received from the electronic device (100) to the external display device.

[0051] In the present disclosure, an electronic device (100) will later describe a method for translating in real time using a window corresponding to a first language that stores data of a first language and a window corresponding to a second language that stores data of a second language.

[0052] Meanwhile, for the convenience of explanation, the language of the text data (subtitle text data) included in the image data acquired by the electronic device (100) is defined as the 'first language' and the target language as the 'second language' and will be described later.

[0053] As illustrated in FIG. 1, the electronic device (100) can acquire first text and audio data in a first language (e.g., Korean) such as “The King of Spain is now said to be Felipe VI.”

[0054] At this time, the electronic device (100) can obtain a first token corresponding to the first text “The King of Spain is now said to be Felipe VI”.

[0055] "Token" may refer to a unit into which text data is divided into small units. For convenience of explanation in this disclosure, a "word" unit constituting text data is defined as a token and described below; however, this is merely one embodiment, and it is understood that it may be a subword (a small unit constituting a word) or a character. At this time, the electronic device (100) may represent the token as an integer after dividing the text data into small units.

[0056] For example, the first token can be represented by integer values ​​of [23525, 45647, 23423, 456232, 5452]. The electronic device (100) can store the first token in a first buffer. The first buffer can store the token converted from text of the first language.

[0057] At this time, the electronic device (100) can store the first token in a window corresponding to the first language. The 'window corresponding to the first language' can be a token among the first tokens to be input into the translation model.

[0058] The electronic device (100) can identify that the length of the window corresponding to the first language is shorter than the length of the multiple tokens corresponding to ‘the King of Spain is now said to be Felipe VI.’ At this time, when the electronic device (100) stores the first token in the window corresponding to the first language, it can delete the tokens in the first part among the multiple tokens included in the first token.

[0059] Meanwhile, when the electronic device (100) updates a window corresponding to the first language, the example below is based on deleting the first token among the multiple tokens included in the first token, but this is only one example, and it is of course possible to delete two or three or more tokens.

[0060] The electronic device (100) can obtain a second token of a second language by inputting a plurality of tokens stored in a window corresponding to a first language into a translation model. For example, the electronic device (100) can obtain a second token corresponding to English by inputting a first token corresponding to Korean into a translation model.

[0061] For example, the second token can be represented by integer values ​​of [532, 753, 2345, 28543, 23256]. The electronic device (100) can store the acquired tokens of [532, 753, 2345, 28543, 23256] in the second buffer.

[0062] The electronic device (100) can identify whether some of the tokens among the acquired second tokens and a plurality of tokens already stored in a window corresponding to the second language match. If the electronic device (100) identifies that some of the tokens among the second tokens match, it can store the matched plurality of tokens in a third buffer.

[0063] At this time, the second buffer can store multiple tokens included in the second token, and the third buffer can store tokens identified as being matched more than a preset number of times among the multiple tokens stored in the second buffer.

[0064] The electronic device (100) can convert a plurality of tokens identified as matching into text of a second language using the decoder part of the translation model.

[0065] For example, as illustrated in FIG. 1, the electronic device (100) can output “The king of Spain is Felipe VI” converted into text in a second language through a display (not shown) or a speaker (not shown).

[0066] The electronic device (100) can update the window corresponding to the second language after storing the token identified as matching in the third buffer by the method described above. Specifically, the electronic device (100) can identify that the length of the plurality of tokens stored in the second buffer and the third buffer is longer than the length of the window corresponding to the second language. At this time, the electronic device (100) can update (or move to the right) the window corresponding to the second language by removing and storing the front token among the plurality of tokens in the second token.

[0067] Meanwhile, for the sake of convenience in the explanation above, the translation model was described in detail based on a model grounded in a transformer structure. However, this is merely one embodiment, and includes at least one of various types of neural network models such as CNN (Convolutional Neural Network), 1DCNN (1-Dimensional Convolutional Neural Network), R-CNN (Region with Convolutional Neural Network), RPN (Region Proposal Network), RNN (Recurrent Neural Network), S-DNN (Stacking-based Deep Neural Network), S-SDNN (State-Space Dynamic Neural Network), Deconvolution Network, DBN (Deep Belief Network), RBM (Restricted Boltzmann Machine), Fully Convolutional Network, LSTM (Long Short-Term Memory) Network, Bi-LSTM (Bidirectional-Long Short-Term Memory) Network, Classification Network, Plain Residual Network, Dense Network, Hierarchical Pyramid Network, Fully Convolutional Network, SENet (Squeeze and Excitation Network), Transformer Network, Encoder, Decoder, Autoencoder, or a combination thereof. Artificial neural networks in this disclosure may be used, and are not limited to the examples described above.

[0068] Additionally, for the sake of convenience of explanation, the following description is based on the premise that the translation model is stored in the memory (120) of the electronic device (100), but this is merely one example and it is obvious that it can be stored in an external device.

[0069] According to the present disclosure, when an electronic device (100) translates text of a first language into text of a second language, it can update a window corresponding to the first language and a window corresponding to the second language in real time. Accordingly, the electronic device (100) can provide the translated text in real time.

[0070] Meanwhile, when inputting text of the first language and outputting text of the second language, any one of the following interfaces may be used: HDMI (High Definition Multimedia Interface), MHL (Mobile High-Definition Link), USB (Universal Serial Bus), DP (Display Port), Thunderbolt, VGA (Video Graphics Array) port, RGB port, D-SUB (D-subminiature), or DVI (Digital Visual Interface). Depending on the implementation example, a port for inputting and outputting only audio signals and a port for inputting and outputting only video signals may be included as separate ports, or a single port for inputting and outputting both audio and video signals may be implemented. The electronic device (100) may transmit at least one of the audio and video signals to an external device (e.g., an external display device or an external speaker) using one of the aforementioned multiple interfaces. At this time, an output port included in one of the multiple interfaces may be connected to the external device, and the electronic device (100) may transmit at least one of the audio data and video data to the external device through the output port.

[0071] FIG. 2 is a block diagram for explaining the configuration of an electronic device (100). The configuration shown in FIG. 2 is merely an example of various embodiments, and some configurations may be omitted, and new configurations may be added. As shown in FIG. 2, the electronic device (100) may include a memory (120) and a processor (120). The configuration shown in FIG. 2 is merely an example, and it goes without saying that some configurations may be deleted or added depending on the configuration of the electronic device (100).

[0072] First, the communication interface (110) is configured to communicate with various types of external devices according to various types of communication methods. In particular, the communication interface (110) can transmit feature values ​​and key values ​​to the server device (200). Alternatively, the communication interface (110) may receive feature values ​​and key values ​​from the server device (200).

[0073] A wireless communication module may be a module that communicates wirelessly with an external device. For example, the wireless communication module may include at least one module among a Wi-Fi module, a Bluetooth module, an infrared communication module, an Ultra Wide-Band (UWB) module, or other communication modules.

[0074] Specifically, the wireless communication module may include at least one communication chip that performs communication according to various wireless communication standards, such as Zigbee, USB (Universal Serial Bus), MIPI CSI (Mobile Industry Processor Interface Camera Serial Interface), 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), and 5G (5th Generation), in addition to the communication method described above. However, this is merely one embodiment, and the communication interface (110) may use at least one communication module among the various communication modules.

[0075] A wired communication module may be a module that communicates with an external device via a wire. For example, a wired communication module may include at least one of a Local Area Network (LAN) module, an Ethernet module, a pair cable, a coaxial cable, or a fiber optic cable.

[0076] According to one embodiment, the communication interface (110) may use the same communication module (e.g., Wi-Fi module) to communicate with an external device, such as a remote control device, and an external server.

[0077] According to one embodiment, the communication interface (110) may use different communication modules to communicate with external devices, such as a remote control device, and external servers. For example, the communication interface (110) may use at least one of an Ethernet module or a Wi-Fi module to communicate with an external server, and may use a Bluetooth module to communicate with an external device, such as a remote control device. However, this is merely one embodiment, and the communication interface (110) may use at least one of various communication modules when communicating with multiple external devices or external servers.

[0078] Memory (120) can store instructions or data related to an operating system (OS) for controlling the overall operation of the components of the electronic device (100) and the components of the electronic device (100). In particular, memory (120) may store information regarding a window corresponding to a first language and a window corresponding to a second language. Additionally, memory (120) may store information regarding a method for updating the window corresponding to the first language and the window corresponding to the second language, and information regarding a first buffer, a second buffer, and a third buffer. Memory (120) may store information regarding a method for determining some tokens among the second tokens of the second language, and may store a translation model.

[0079] Memory (120) can be implemented in various forms such as volatile memory (e.g., DRAM (dynamic RAM), SRAM (static RAM), or SDRAM (synchronous dynamic RAM), etc.), non-volatile memory (e.g., OTPROM (one time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), etc.), hard drive, or solid state drive (SSD).

[0080] The display (130) can display various information. In particular, the display (130) can also display translated text.

[0081] The display (130) can be implemented as various types of displays such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, and a PDP (Plasma Display Panel). The display may also include a driving circuit, a backlight unit, etc., which can be implemented in forms such as an a-si TFT (amorphous silicon thin film transistor), an LTPS (low temperature poly silicon) TFT, and an OTFT (organic TFT). The display can be implemented as a touch screen combined with a touch sensor, a flexible display, a 3D display, a three-dimensional display, etc. According to various embodiments of the present disclosure, the display (130) may include not only a display panel that outputs an image, but also a bezel that houses the display panel.

[0082] The speaker (140) is configured to output various audio data, as well as various notification sounds or voice messages, after various processing operations such as decoding, amplification, and noise filtering have been performed. In particular, the speaker (140) can output information about an event in the form of audio. At this time, the speaker (140) may be provided in multiple areas of the electronic device (e.g., the upper area of ​​the front of the electronic device or the lower side area of ​​the electronic device, etc.).

[0083] In particular, the speaker (140) can output a voice signal of the source language (or, first language) or convert text information of the target language (or, second language) into a voice signal and output it.

[0084] The processor (150) may include one or more processors. Specifically, one or more processors may include one or more of a CPU (Central Processing Unit), GPU (Graphics Processing Unit), APU (Accelerated Processing Unit), MIC (Many Integrated Core), DSP (Digital Signal Processor), NPU (Neural Processing Unit), hardware accelerator, or machine learning accelerator. The processor (120) may control one or any combination of other components of the electronic device and may perform operations or data processing related to communication. One or more processors may execute one or more programs or instructions stored in memory. For example, one or more processors may perform a method according to one embodiment of the present disclosure by executing one or more instructions stored in memory.

[0085] One or more processors may be implemented as a single-core processor comprising one core, or as one or more multicore processors comprising multiple cores (e.g., homogeneous multicore or heterogeneous multicore). When one or more processors are implemented as multicore processors, each of the multiple cores included in the multicore processor may include internal processor memory such as cache memory or on-chip memory, and a common cache shared by multiple cores may be included in the multicore processor. Additionally, each of the multiple cores included in the multicore processor (or some of the multiple cores) may independently read and execute program instructions for implementing a method according to one embodiment of the present disclosure, or all (or some) of the multiple cores may be linked together to read and execute program instructions for implementing a method according to one embodiment of the present disclosure.

[0086] In particular, when a first text of a first language is input, the processor (120) can acquire a first token corresponding to the first text language and store it in a window corresponding to the first language, update the window corresponding to the first language based on the length of the first token, input a plurality of tokens stored in the updated window corresponding to the first language into a translation model to acquire a second token of the second language, identify whether some of the second tokens of the second language and the plurality of tokens already stored in the window corresponding to the second language match, and based on the identification result, determine some of the second tokens of the second language and update the window corresponding to the second language.

[0087] Additionally, the processor (120) may acquire a first token corresponding to a first text of a first language that has been input, update a window corresponding to the first language in which the first token is stored based on the length of the first token, and acquire a second token of the second language through an artificial intelligence model based on a plurality of tokens stored in the updated window corresponding to the first language. Additionally, if there is at least one token included in the acquired second token of the second language corresponding to a plurality of tokens already stored in the window corresponding to the second language, the processor (120) may update the window corresponding to the second language based on at least one token.

[0088] Additionally, the processor (120) may store the acquired first token in the first buffer and update a window corresponding to the first language, which identifies the number of stored tokens according to the first language, based on the number of tokens included in the first token.

[0089] If the length of the first token is longer than the length of the window corresponding to the first language, the processor (120) may update by storing some tokens included in the first token in the window corresponding to the first language. The processor (120) may remove the front token among the first tokens and update by storing the added token among the multiple tokens stored in the window corresponding to the first language.

[0090] The processor (120) can store a second token of the second language in a second buffer and identify whether a plurality of tokens included in the second token of the second language correspond to a plurality of tokens stored in a window corresponding to the second language more than a preset number of times.

[0091] The processor (120) may store at least one token included in the second token of the acquired second language in the third buffer.

[0092] If the length of the second token of the second language is longer than the window length corresponding to the second language, the processor (120) may update by removing the token with the highest priority among the multiple tokens included in the second token. The window corresponding to the second language may store multiple tokens stored in a second buffer and a third buffer in which at least one token included in the acquired second token of the second language is stored.

[0093] The processor (120) may set the length of a window corresponding to the first language based on the first language, and set the length of a window corresponding to the second language based on the second language.

[0094] If the processor (120) identifies that there is at least one token included in the second token of the acquired second language corresponding to a plurality of tokens already stored in a window corresponding to the second language, the processor (120) may determine at least one token. Additionally, the processor (120) may update the window corresponding to the second language containing at least one determined token. Meanwhile, each of the steps described above will be described below with reference to FIGS. 3 to 8.

[0095] FIG. 3 is a diagram illustrating a process of performing a translation operation in real time using a window corresponding to a first language and a window corresponding to a second language, according to one embodiment of the present disclosure.

[0096] First, as shown in FIG. 3, when a first text of a first language is input, the electronic device (100) can acquire a first token corresponding to the first text language and then store it in a window corresponding to the first language.

[0097] At this time, the electronic device (100) can update a window corresponding to the first language based on the length of the first token. This will be described later with reference to FIGS. 4 and FIGS. 5.

[0098] As shown in FIG. 4, the electronic device (100) can obtain a first token corresponding to a first text, “Now the King of Spain”. At this time, as shown in FIG. 5, the electronic device (100) can identify a plurality of tokens corresponding to the text “Now the King of Spain” as [83, 142, 165, 15, 18, 253, 987, 264, 724].

[0099] The electronic device (100) can store the acquired first token in a first buffer and a window corresponding to a first language. If the electronic device (100) identifies that the length of the first token is shorter than the length of the window corresponding to the first language, it can update the first token by storing it in the window corresponding to the first language.

[0100] Meanwhile, the electronic device (100) can identify that the length of the acquired first token is longer than the length of the window corresponding to the first language. At this time, the electronic device (100) stores the first token in the first buffer, sets the number of tokens that can be stored in the window corresponding to the first language according to the first language, and can update the window corresponding to the first language based on the number of tokens included in the first token. Additionally, if the electronic device (100) identifies that the length of the first token is longer than the window of the source language (the window corresponding to the source language), it can update the source language window by storing some of the tokens of the first token. When updating the source language window, the electronic device (100) can remove the front tokens of the first token and update the source language window by storing the added tokens among the multiple tokens stored in the source language window.

[0101] For example, the electronic device (100) can obtain a first token corresponding to a first text, "it is said that the King of Spain is Felipe VI." Additionally, the electronic device (100) can obtain a text in a first language, "it is said that the King of Spain is Felipe VI." At this time, the electronic device (100) can identify multiple tokens corresponding to the text, "it is said that the King of Spain is now Felipe VI," as [83, 142, 165, 15, 18, 253, 987, 264, 724, 523, 6465, 245, 2452, 69, 2, 624].

[0102] The electronic device (100) can identify that the acquired first token is longer than the length of the window corresponding to the first language. At this time, the electronic device (100) can remove the tokens in the front part of the first language tokens and store the tokens in the back part of the window corresponding to the first language.

[0103] For example, if the electronic device (100) identifies that the length of [83, 142, 165, 15, 18, 253, 987, 264, 724, 523, 6465, 245, 2452, 69,2, 624] is longer than the length of the window corresponding to the first language, it may remove [83, 142, 165, 15, 18, 253, 987] from the preceding tokens and update the tokens of [523, 6465, 245, 2452, 69,2, 624].

[0104] After acquiring a first token, the electronic device (100) can acquire a second token corresponding to a second language. At this time, the electronic device (100) can use the second token to update a window corresponding to a second language and determine some of the second tokens of the second language.

[0105] Specifically, the electronic device (100) stores a second token of the second language in a second buffer and can identify that some of the second tokens of the second language and a plurality of tokens stored in the window of the target language match more than a preset number of times. Additionally, the electronic device (100) can store some of the identified second tokens of the second language in a third buffer. If the electronic device (100) identifies that the length of the second token of the second language is greater than the window size of the target language, it may update by removing the first token among the plurality of tokens included in the second token.

[0106] Specifically, among the multiple tokens included in the second token, the token located first may represent a token with high priority. In this case, 'priority' may represent the degree of high probability of being confirmed among the multiple tokens included in the second token. Additionally, priority may represent the similarity value between tokens. This will be described later with reference to FIGS. 6 and 7.

[0107] The electronic device (100) can determine the token in the decoder included in the translation model using the first token obtained as described above in FIGS. 4 and FIGS. 5.

[0108] As illustrated in FIG. 6, the electronic device (100) may store a second token of a second language in a second buffer. The second token may be a token corresponding to "The king of Spain is Felipe VI, and he's popular. He's popular with the people". Specifically, as illustrated in FIG. 7, the second token may be [624, 921, 702, 322, 624, 4643, 234, 532, 533, 6432].

[0109] The electronic device (100) can identify whether some of the second tokens of the second language and a plurality of tokens already stored in the second buffer match.

[0110] For example, the electronic device (100) can identify whether a plurality of tokens [624, 921, 702] and [624, 921, 702, 322, 624, 4643, 234, 532, 533, 6432], which are stored in the second buffer, are matched. At this time, the electronic device (100) can identify that [624, 921, 702] are matched.

[0111] At this time, the electronic device (100) may store a plurality of tokens identified as matching in a third buffer. Specifically, the plurality of tokens identified as matching may be [624, 921, 702].

[0112] Meanwhile, for the sake of convenience of explanation, an embodiment has been described in detail in which a portion of the second token identified as being matched is determined when a portion of the second token corresponding to a previously stored second language and a portion of the second language are matched once. However, this is merely one embodiment, and it is obvious that a portion of the second token can be identified as being matched when it is identified as being matched two or more times for the accuracy of translation.

[0113] After determining some of the tokens among the second tokens, the electronic device (100) can identify whether the length of the tokens stored in the second buffer and the third buffer is longer than the window corresponding to the second language.

[0114] If the electronic device (100) identifies that the length of the tokens stored in the second buffer and the third buffer is longer than the window corresponding to the second language, it may store some of the tokens among the plurality of tokens included in the second token in the window corresponding to the second language.

[0115] For example, the electronic device (100) can update a window corresponding to the second language by removing the first token [624, 921, 702, 532] of the second token [624, 921, 702, 322, 624, 4643, 234, 532, 533, 6432] and saving the second token anew.

[0116] FIG. 8 is a drawing for explaining an embodiment of outputting after confirming a plurality of tokens among the second tokens according to one embodiment of the present disclosure.

[0117] As illustrated in FIG. 8, the electronic device (100) can identify the text of a second language (or target language) corresponding to the confirmed token [624, 921, 702] as “The king of Spain is Felipe VI” and then output it through a display (not shown) or a speaker (not shown).

[0118] FIG. 9 is a flowchart illustrating a method for controlling an electronic device according to one embodiment of the present disclosure.

[0119] The electronic device (100) can obtain a first token corresponding to a first text of the input first language (S910).

[0120] The electronic device (100) can update the window corresponding to the first language in which the first token is stored based on the length of the first token (S920). Specifically, if the electronic device (100) identifies that the length of the first token is shorter than the length of the window corresponding to the first language, it can store the first token without any separate modification. On the other hand, if the electronic device (100) identifies that the length of the first token is longer than the length of the window corresponding to the first language, it can store the first token, including the latter part excluding the former part of the first token, in the window corresponding to the first language.

[0121] The electronic device (100) can obtain a second token of a second language through an artificial intelligence model based on a plurality of tokens stored in a window corresponding to an updated first language (S930). At this time, the translation model may include a transformer structure, and since this has been described above, a detailed explanation will be omitted.

[0122] Additionally, if there is at least one token included in the second token of the acquired second language corresponding to a plurality of tokens stored in the window corresponding to the second language, the electronic device (100) can update the window corresponding to the second language based on at least one token (S940).

[0123] Specifically, the electronic device (100) can determine some of the second tokens of the second language identified as matching when a plurality of tokens stored in a window corresponding to the second language and some of the second tokens of the second language match. Additionally, the electronic device (100) can store the some tokens identified as being determined in a third buffer, and store the some tokens that are not determined in a second buffer.

[0124] The electronic device (100) can determine some of the second tokens of the second language based on the identification result and update the window corresponding to the second language (the window of the target language or the window corresponding to the target language) (S950).

[0125] Specifically, if the electronic device (100) identifies that the number of tokens in the second buffer and the third buffer is greater than the size of the window corresponding to the second language, it can remove the token at the very front of the second buffer and the third buffer.

[0126] Additionally, methods according to various embodiments of the present disclosure may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user (20) devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., downloadable app) may be temporarily stored or temporarily created in a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0127] A method according to various embodiments of the present disclosure may be implemented as software comprising instructions stored on a machine-readable storage medium (e.g., a computer). The machine may include a server device or an electronic device according to the disclosed embodiments, which is a device capable of calling instructions stored from the storage medium and operating according to the called instructions.

[0128] Meanwhile, a device-readable storage medium may be provided in the form of a non-transitory readable recording medium. Here, 'non-transitory readable recording medium' simply means that it is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily. For example, a 'non-transitory storage medium' may include a buffer in which data is stored temporarily.

[0129] When the above instruction is executed by a processor, the processor may perform the function corresponding to the instruction directly or by using other components under the control of the processor. The instruction may include code generated or executed by a compiler or an interpreter.

[0130] Although preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure.

[0131] 100 Electronic device 110 Communication interface

[0132] 120 memory 130 display

[0133] 140 speakers 150 processors

Claims

1. In an electronic device, Memory for storing instructions; and Includes at least one processor; and When the above instructions are executed collectively or individually by the at least one processor, the electronic device, Obtain a first token corresponding to the input text of the first language, and Based on the length of the first token, update the window corresponding to the first language where the first token is stored, and Based on a plurality of tokens stored in a window corresponding to the above-mentioned updated first language, a second token of the second language is obtained through an artificial intelligence model, and An electronic device that updates a window corresponding to the second language based on at least one token if there is at least one token included in the second token of the acquired second language corresponding to a plurality of tokens stored in a window corresponding to the second language.

2. In Paragraph 1, When the above instructions are executed collectively or individually by the at least one processor, the electronic device, The above-mentioned first token is stored in the first buffer, and An electronic device that identifies the number of storage tokens according to the first language and updates a window corresponding to the first language based on the number of tokens included in the first token.

3. In Paragraph 2, When the above instructions are executed collectively or individually by the at least one processor, the electronic device, An electronic device that, if the length of the first token is longer than the length of the window corresponding to the first language, stores some tokens included in the first token in the window corresponding to the first language to update the window corresponding to the first language.

4. In Paragraph 3, The first token above is one of a plurality of tokens of the first language, and When the above instructions are executed collectively or individually by the at least one processor, the electronic device, Remove the front token among the plurality of tokens of the first language above, and An electronic device that updates a window corresponding to the first language by storing an added token among a plurality of tokens stored in a window corresponding to the first language.

5. In Paragraph 1, The second token above is one of a plurality of tokens of the second language, and When the above instructions are executed collectively or individually by the at least one processor, the electronic device, The second token of the above second language is stored in the second buffer, and An electronic device that identifies whether a plurality of tokens of the second language correspond to a plurality of tokens stored in a window corresponding to the second language more than a predetermined number of times.

6. In Paragraph 5, When the above instructions are executed collectively or individually by the at least one processor, the electronic device, An electronic device that stores at least one token included in a plurality of tokens of the second language in a third buffer.

7. In Paragraph 1, The second token above is one of a plurality of tokens of the second language, and When the above instructions are executed collectively or individually by the at least one processor, the electronic device, An electronic device that updates by removing a token with a higher priority among a plurality of tokens of the second language when the length of the second token of the second language is longer than the window length corresponding to the second language.

8. In Paragraph 1, The second token above is one of a plurality of tokens of the second language, and An electronic device in which a window corresponding to the second language stores a plurality of tokens stored in the second buffer and a third buffer in which at least one of the plurality of tokens of the second language is stored.

9. In Paragraph 1, When the above instructions are executed collectively or individually by the at least one processor, the electronic device, Based on the first language, the length of the window corresponding to the first language is set, and An electronic device that sets the length of a window corresponding to the second language based on the second language.

10. In Paragraph 1, When the above instructions are executed collectively or individually by the at least one processor, the electronic device, An electronic device that updates a window corresponding to the second language containing at least one acquired token if there is at least one token included in the second token of the acquired second language corresponding to a plurality of tokens stored in a window corresponding to the second language.

11. In a method for controlling an electronic device, A step of obtaining a first token corresponding to the input text of the first language; A step of updating a window corresponding to the first language in which the first token is stored, based on the length of the first token; A step of obtaining a second token of a second language through an artificial intelligence model based on a plurality of tokens stored in a window corresponding to the above-mentioned updated first language; and A control method comprising the step of updating a window corresponding to the second language based on at least one token if there is at least one token included in the second token of the acquired second language corresponding to a plurality of tokens stored in a window corresponding to the second language.

12. In Paragraph 11, A step of storing the first token obtained above in a first buffer; and A control method comprising: a step of updating a window corresponding to the first language, which identifies the number of storage tokens according to the first language, based on the number of tokens included in the first token.

13. In Paragraph 12, A control method comprising the step of, if the length of the first token is longer than the length of the window corresponding to the first language, storing some tokens included in the first token in the window corresponding to the first language to update the window corresponding to the first language.

14. In Paragraph 13, The first token above is one of a plurality of tokens of the first language, and A step of removing the front token among the plurality of tokens of the first language; and A control method comprising the step of updating the window corresponding to the first language by storing an added token among a plurality of tokens stored in a window corresponding to the first language.

15. In Paragraph 11, The second token above is one of a plurality of tokens of the second language, and A step of storing the second token of the second language in the second buffer; and A control method comprising: a step of identifying whether a plurality of tokens of the second language correspond to a plurality of tokens stored in a window corresponding to the second language more than a predetermined number of times.

Images