Electronic device and method of operating the same

Abstract

An electronic device is disclosed, comprising: a camera configured to capture an image of at least one object; a display; a memory for storing one or more instructions; and a processor configured to execute the one or more instructions stored in the memory to: recognize at least one text included in the image received from the camera, generate one or more recommendation words with priority based on the recognized at least one text and context information, and control the display to display the generated one or more recommendation words according to the priority.

Description

Technical Field

[0001] Various embodiments relate to an electronic device and an operating method thereof, and more particularly, to an electronic device capable of providing at least one recommendation word based on at least one text recognized in an image and an operating method thereof. Background Art

[0002] With the advancement of information communication technology and semiconductor technology, various electronic devices are developing into multimedia devices that provide various multimedia services. For example, electronic devices provide various multimedia services such as messenger services, broadcast services, wireless Internet services, camera services, and music playback services.

[0003] Specifically, the electronic device enables easy imaging of the surrounding environment by using a camera. In addition, the electronic device can output a result of recognizing characters included in an image by using an optical character recognition (OCR) technology for automatically recognizing characters in an image.

[0004] However, when text is input by recognizing characters included in an image using an OCR function while the text is being input, it is troublesome, and there is a disadvantage in that the result of character recognition using OCR is not satisfactory. Summary of the invention

[0005] Technical issues

[0006] Various embodiments may provide an electronic device and an operating method thereof, the electronic device being capable of recognizing text in an image and generating and displaying recommendation words based on the recognized text and context information.

[0007] Solution to the problem

[0008] According to an embodiment, an electronic device includes: a camera configured to capture an image of at least one object; a display; a memory storing one or more instructions; and a processor configured to execute the one or more instructions stored in the memory to identify at least one text included in the image received from the camera, generate one or more recommendation words with priority based on the identified at least one text and context information, and control the display to display the generated one or more recommendation words according to the priority.

[0009] According to an embodiment, an operating method of an electronic device may include: obtaining an image by capturing an image of at least one object; identifying at least one text included in the image; generating one or more recommendation words with priority based on the identified at least one text and context information; and displaying the generated one or more recommendation words according to the priority.

[0010] Advantageous Effects of the Present Disclosure

[0011] According to an embodiment, the electronic device can input words only by capturing an image instead of typing in a text input mode, so that the user's convenience of use can be enhanced.

[0012] According to an embodiment, the electronic device may recommend words matching the user's intention by changing the recognized text into words matching the context information. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a diagram showing an electronic device according to an embodiment.

[0014] Figure 2 is a block diagram showing a configuration of an electronic device according to an embodiment.

[0015] Figure 3 It is shown Figure 2 A block diagram of a configuration of a processor.

[0016] Figure 4 is a flowchart illustrating an operating method of an electronic device according to an embodiment.

[0017] Figures 5 to 10 is a diagram for reference in describing an operating method of an electronic device according to an embodiment.

[0018] Fig.11 is a block diagram showing a configuration of an electronic device according to another embodiment. DETAILED DESCRIPTION

[0019] According to an embodiment, an electronic device includes: a camera configured to capture an image of at least one object; a display; a memory storing one or more instructions; and a processor configured to execute the one or more instructions stored in the memory, wherein by executing the one or more instructions, the processor is further configured to recognize at least one text included in the image received from the camera, generate one or more recommendation words with priority based on the recognized at least one text and context information, and control the display to display the generated one or more recommendations according to the priority.

[0020] According to an embodiment, the processor may also be configured to: when the electronic device is operating in a text input mode, control the display to display a text input window and a capture icon; and when receiving user input selecting the capture icon, drive the camera and control the camera to capture an image of at least one object.

[0021] According to an embodiment, the processor may be further configured to change the recognized at least one text into a word matching the context information by using a language model.

[0022] According to an embodiment, the processor may also be configured to determine the priority of the at least one identified text based on at least one of the position of the at least one text in the image, the size of the at least one text, the color of the at least one text, or the type of the at least one text.

[0023] According to an embodiment, the processor may be further configured to determine a priority of the one or more recommended words based on the recognized category of the at least one text and context information of the text input to the electronic device.

[0024] According to an embodiment, when the electronic device is in a state of sending and receiving messages with an external device, the context information may include at least one of time information of the electronic device, location information of the electronic device, information about the external device, content of a message received from the external device, or content of a message input into the message input area.

[0025] According to an embodiment, the processor may be further configured to control the display to display the generated one or more recommendation words on the display in order starting from the highest priority.

[0026] According to an embodiment, the processor may be further configured to, upon receiving a user input selecting any one of the generated one or more recommendation words, control inputting the selected recommendation word into the text input window.

[0027] According to an embodiment, an operating method of an electronic device may include: obtaining an image by capturing an image of at least one object; identifying at least one text included in the image; generating one or more recommendation words with priority based on the identified at least one text and context information; and displaying the generated one or more recommendation words according to the priority.

[0028] According to an embodiment, the method may further include: when the electronic device is operating in a text input mode, displaying a text input window and a capture icon, wherein obtaining an image by capturing an image of at least one object includes: when a user input selecting the capture icon is received, driving a camera and capturing an image of at least one object.

[0029] According to an embodiment, generating one or more recommended words with priorities may include changing the recognized at least one text into words matching the context information by using a language model.

[0030] According to an embodiment, identifying at least one text included in an image may include: determining the priority of the identified at least one text based on at least one of a position of the at least one text in the image, a size of the at least one text, a color of the at least one text, or a type of the at least one text.

[0031] According to an embodiment, generating one or more recommendation words with priorities may include determining the priorities of the one or more recommendation words based on the recognized category of the at least one text and context information of the text input to the electronic device.

[0032] According to an embodiment, when the electronic device is in a state of sending and receiving messages with an external device, the context information may include at least one of time information of the electronic device, location information of the electronic device, information about the external device, content of a message received from the external device, or content of a message input into the message input area.

[0033] According to an embodiment, displaying the generated one or more recommendation words according to the priority may include displaying the one or more recommendation words on the display in order starting from the highest priority.

[0034] According to an embodiment, the method may further include receiving a user input selecting any one of the one or more recommendation words, and inputting the selected recommendation word into the text input window.

[0035] Embodiments of the present disclosure

[0036] Terms used in the specification will be briefly described, and the present disclosure will be described in detail.

[0037] The terms used in the present disclosure have been selected while taking into account the functions in the present disclosure, and the currently widely used general terms have been selected, but this may change according to the intention of those skilled in the art or the emergence of precedents or new technologies. In addition, in some cases, the applicant arbitrarily selects some terms, and in this case, their meanings will be described in detail in the applicable description of the present disclosure. Therefore, the terms used in the present disclosure should be defined based on the meaning of the terms and the entire content of the present disclosure rather than simple term names.

[0038] When a part of the specification "includes" a certain component, it means that other components may be further included rather than excluded, unless there is an explicit description to the contrary. In addition, terms such as "unit" and "module" described in the specification refer to a unit that processes at least one function or operation, which can be implemented in hardware or software, or a combination of hardware and software.

[0039] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present disclosure belongs can easily practice the embodiments. However, the present disclosure can be implemented in many different forms and is not limited to the embodiments described herein. In addition, in order to clearly describe the present disclosure in the accompanying drawings, parts not related to the description are omitted, and similar reference numerals are assigned to similar parts throughout the specification.

[0040] Figure 1is a diagram showing an electronic device according to an embodiment.

[0041] According to an embodiment, the electronic device 100 can be implemented in various forms. For example, the electronic device can be implemented in various electronic devices, including mobile phones, smart phones, laptop computers, desktop computers, tablet personal computers (PCs), e-book terminals, digital broadcast terminals, personal digital assistants (PDAs) and portable multimedia players (PMPs), navigation, MP3 players, portable cameras, Internet Protocol televisions (IPTVs), digital televisions (DTVs), and wearable devices. However, the electronic device 100 is not limited thereto.

[0042] In one or more embodiments, the term "user" refers to a person who controls the function or operation of an electronic device, which may include an administrator or an installation engineer.

[0043] According to an embodiment, the electronic device 100 may include a camera that captures an image of at least one object. Figure 1 As shown, the electronic device 100 can perform image capture by using a camera in a text input mode. For example, the text input mode may include a mode for inputting text to send a text message or an instant message to an external device, a mode for inputting text by running an application capable of inputting text (such as a memo application), and a mode for inputting text to send an email. However, the text input mode is not limited to this. According to an embodiment, the electronic device 100 can display a text input window and a camera icon in the text input mode.

[0044] In the text input mode, when receiving a user input requesting image capture (eg, input of touching a camera icon), the electronic device 100 may drive the camera to capture an image of at least one object and display the camera-captured image as a preview image.

[0045] In addition, the electronic device 100 may include an optical character recognition (OCR) module, and may perform an OCR function by using the OCR module. The OCR function is used to read characters by using light, which is a function of recognizing at least one text included in an image. According to an embodiment, the electronic device 100 may recognize at least one text included in an image captured by a camera by using the OCR function.

[0046] In addition, the electronic device 100 can be based on the similarity between the shape of the recognized text and the shape of the text before recognition, the position of the recognized text in the image, the size of the text, the orientation of the text, the color of the text, the font, whether the recognized text is printed text or handwritten text, the language type in the text, the category of the text, etc. However, the embodiment is not limited to this.

[0047] For example, the electronic device 100 may set the highest priority for the text recognized at the position closest to the center point of the image in the recognized text. Alternatively, the electronic device 100 may set the highest priority for the text of the largest size in the recognized text. Alternatively, the electronic device 100 may set the highest priority for the text whose orientation is closest to the horizontal orientation in the recognized text. However, the embodiment is not limited to this.

[0048] According to an embodiment, the electronic device 100 may generate one or more recommended words by changing the recognized text into words matching the context information using a language model. In addition, the electronic device 100 may determine the priority of the recommended words by using a language model.

[0049] A language model may be a model that models the language used by humans to calculate the degree to which a specific word string corresponds to the language actually used as a probability. A language model may be a model that predicts grammatically appropriate or highly probable words or letters based on a dictionary, frequency of use, probability of use, etc. For example, in the case of a language model of a character or word unit, the language model may predict the next letter or word based on the letters or words input so far. In addition, in the case of a language model of a sentence unit, the language model may predict the words included in the sentence, taking into account the comprehensive meaning or context of the sentence or paragraph.

[0050] The language model may be a model based on a neural network. For example, a model such as a deep neural network (DNN), a recurrent neural network (RNN), or a bidirectional recurrent deep neural network (BRDNN) may be used as a language model, but is not limited thereto.

[0051] In addition, the context information may include time information of the electronic device, location information of the electronic device, information about the other party of the message transmission and reception, content of the message received from the other party, and content input to the message input area. However, the context information is not limited thereto.

[0052] According to an embodiment, the electronic device 100 may change at least one text recognized in the image into one or more words matching the context information by using a language model. In addition, the electronic device 100 may re-determine the priority of one or more words by using a language model.

[0053] According to an embodiment, the electronic device 100 may display the recommendation words on the display according to the priority, and may display the recommendation words in order starting from the highest priority.

[0054] Figure 2 is a block diagram showing a configuration of an electronic device 100 according to an embodiment.

[0055] refer to Figure 2According to an embodiment, the electronic device 100 may include a camera 110 , a processor 120 , a memory 130 , and a display 140 .

[0056] According to an embodiment, the camera 110 may obtain an image frame such as a still image or a video through an image sensor. The image captured by the image sensor may be processed by the processor 120 or a separate image processor (not shown). The image frame processed in the camera 110 may be stored in the memory 130 or sent to an external device. According to the configuration of the electronic device 100, two or more cameras 110 may be provided.

[0057] According to an embodiment, the camera 110 may capture an image of at least one object including text, and process the captured image by using the processor 120 .

[0058] According to an embodiment, the processor 120 may run one or more programs stored in the memory 130. The processor 120 may include a single core, a dual core, a triple core, a quad core and / or a multi-core. In addition, the processor 120 may include a plurality of processors. For example, the processor 120 may be implemented as a main processor (not shown) and a sub-processor (not shown) operating in a sleep mode.

[0059] According to an embodiment, the memory 130 may store various data, programs, or applications for driving and controlling the electronic device 100 .

[0060] In addition, the program stored in the memory 130 may include one or more instructions. The program (one or more instructions) or application stored in the memory 130 may be executed by the processor 120.

[0061] According to an embodiment, the processor 120 may recognize at least one text in the image received from the camera 110 by executing one or more instructions stored in the memory 130 .

[0062] For example, the processor 120 can identify the text included in the image by using the OCR function. In addition, the processor 120 can divide the captured image into multiple areas and identify the text by area. At this time, the priority of the multiple areas can be determined based on whether the shape appearing in each area of ​​the multiple areas is similar to the text and the position in each area. In addition, the text identified in each area of ​​the multiple areas with priority can have the same priority as its corresponding area. Alternatively, the processor 120 can determine the priority of the recognized text based on whether the shape of the text appearing in the image is similar to the text, the position of the text in the image, the size of the text, the orientation of the text, the color of the text, the font, whether the recognized text is printed text or handwritten text, the language type in the text, the category of the text, etc. However, the embodiment is not limited to this.

[0063] The processor 120 may generate one or more recommended words with priorities based on the recognized text and context information. For example, the processor 120 may change at least one recognized text into one or more recommended words matching the context information by using a language model. In addition, the processor 120 may re-determine the priority of the words changed into the recommended words.

[0064] According to an embodiment, the display 140 generates a driving signal by converting an image signal, a data signal, an on-screen display (OSD) signal, a control signal, etc. processed by the processor 120. The display 140 may be implemented as a plasma display panel (PDP), a liquid crystal display (LCD), an organic light emitting diode (OLED), a flexible display, etc., and may also be implemented as a three-dimensional (3D) display. In addition, the display 140 may be configured as a touch screen and may be used as an input device in addition to an output device.

[0065] According to an embodiment, the display 140 may display an image captured by the camera 110. In addition, the display 140 may display the generated recommendation words according to priority. For example, the display 140 may display the recommendation words in order starting from a high priority.

[0066] Figure 3 It is shown Figure 2 A block diagram of a configuration of a processor.

[0067] refer to Figure 2 According to some embodiments, the processor 120 may include a data learning unit 121 and a data identification unit 122 .

[0068] The data learning unit 121 may learn a standard for changing text recognized in an image into a word (e.g., a recommended word) that matches the context. The data learning unit 121 may learn a standard for which data to use to generate a recommended word and how to generate the recommended word by using the data. The data learning unit 121 may learn a standard for generating a recommended word by obtaining data to be used for training and applying the obtained data to a language model, which will be described later.

[0069] The data recognition unit 122 may generate a recommendation word based on the data. The data recognition unit 122 may generate a recommendation word from preset data by using a trained language model. The data recognition unit 122 may obtain preset data (e.g., at least one text recognized in an image, context information, etc.) according to a preset standard obtained through learning, and utilize a language model by using the obtained data as an input value, so that a recommendation word may be generated based on the preset data. In addition, a result value output by the language model by using the obtained data as an input value may be used to refine the language model.

[0070] At least one of the data learning unit 121 or the data identification unit 122 may be manufactured in the form of at least one hardware chip and installed on the electronic device. For example, at least one of the data learning unit 121 or the data identification unit 122 may be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI), or may be manufactured as part of an existing general-purpose processor (e.g., a central processing unit (CPU) or an application processor) or only a graphics processor (e.g., a graphics processing unit (GPU)) and installed on the above-mentioned various electronic devices.

[0071] In this case, the data learning unit 121 and the data identification unit 122 may be installed on one electronic device, or may be installed on separate electronic devices, respectively. For example, one of the data learning unit 121 and the data identification unit 122 may be included in the electronic device, and the other may be included in the server. In addition, the data learning unit 121 and the data identification unit 122 may provide the model information constructed by the data learning unit 121 to the data identification unit 122 by wired or wireless means, and may provide the data input to the data identification unit 122 as additional training data to the data learning unit 121.

[0072] Meanwhile, at least one of the data learning unit 121 or the data identification unit 122 may be implemented as a software module. When at least one of the data learning unit 121 or the data identification unit 122 is implemented as a software module (or a program module including instructions), the software module may be stored in a non-transitory computer-readable medium. In addition, in this case, at least one software module may be provided by an operating system (OS) or may be provided by a preset application. Alternatively, some of the at least one software module may be provided by the OS, while others may be provided by a preset application.

[0073] Figure 4 is a flowchart illustrating an operating method of an electronic device according to an embodiment.

[0074] refer to Figure 4 According to an embodiment, in operation S210, the electronic device 100 may capture an image of at least one object. For example, when the electronic device 100 receives a user input for requesting image capture, the electronic device 100 may drive a camera to capture an image of at least one object. At this time, the object captured by the camera may include at least one text. In addition, the electronic device 100 may display the image captured by the camera as a preview image.

[0075] According to an embodiment, in operation S220, the electronic device 100 may identify at least one text included in the captured image. For example, the electronic device 100 may identify the text included in the image by using an OCR function. The electronic device 100 may divide the captured image into a plurality of regions and perform text recognition by region. At this time, the priority of the plurality of regions may be determined based on whether the shape appearing in each of the plurality of regions is similar to the text or the position in each region. In addition, the text recognized in each of the plurality of regions having a priority may have the same priority as its corresponding region. For example, when the captured image is divided into a first region, a second region, and a third region, and the first region has a first priority, the second region has a third priority, and the third region has a second priority, the first text recognized in the first region may have a first priority, the second text recognized in the second region may have a third priority, and the third text recognized in the third region may have a second priority. However, the embodiment is not limited thereto.

[0076] Alternatively, the electronic device 100 can determine the priority of the recognized text based on whether the shape of the text appearing in the image is similar to the text, the position of the text in the image, the size of the text, the orientation of the text, the color of the text, the font, whether the recognized text is printed text or handwritten text, the language type in the text, the category of the text, etc. However, the embodiment is not limited to this. For example, the electronic device 100 can set the highest priority for the text recognized at the position closest to the center point of the image among the recognized texts. Alternatively, the electronic device 100 can set the highest priority for the text with the largest size among the recognized texts. Alternatively, the electronic device 100 can set the highest priority for the text whose orientation is closest to the horizontal orientation among the recognized texts. However, the embodiment is not limited to this.

[0077] According to an embodiment, in operation S230, the electronic device 100 may obtain context information.

[0078] For example, the electronic device 100 can obtain time information of the electronic device, location information of the electronic device, information about the other party of the message sending and receiving, content of the message received from the other party, and content of the message input in the message input area.

[0079] According to an embodiment, in operation S240, the electronic device 100 may generate at least one recommended word with priority by using a language model. For example, the electronic device 100 may change the recognized text into one or more words matching the context information by using a language model. In addition, the electronic device 100 may re-determine the priority of one or more words by using a language model.

[0080] According to an embodiment, in operation S250, the electronic device 100 may display at least one recommendation word on the display according to the priority. For example, the electronic device 100 may display at least one recommendation word in order starting from the highest priority.

[0081] Figures 5 to 10 is a diagram referred to for describing an operating method of an electronic device according to an embodiment.

[0082] refer to Figure 5 , a user of the electronic device (first user) can send and receive messages with a user of the external device (second user), and the electronic device 100 can display a message window 410. According to an embodiment, the message window 410 may include at least one of a text message window, an instant message window, an email window, or a chat window, but is not limited thereto.

[0083] In addition, the message window 410 may include a message input area 430 and a message display area 420. In this case, the message input area 430 may refer to an area for inputting a message to be sent, and the message display area 420 may refer to an area for displaying a message completed to be sent or received.

[0084] For example, the second user may send a message asking where the first user is (eg, where are you?), and the message sent by the second user may be displayed in the message display area 420 .

[0085] At this time, the first user may input the place where he or she is located by capturing an image of a landmark or the like around the first user, instead of typing the place where he or she is located in the message input area 430 .

[0086] like Figure 5 As shown, in addition to the message display area 420 and the message input area 430, the message window 410 may further include a camera icon 450. The camera icon 450 may be displayed near the message input area 430. Alternatively, the electronic device 100 may include a camera button 460, and the camera button 460 may be implemented as a hardware button.

[0087] When receiving an input of touching the capture icon 450 or an input of pressing the capture button 460, the electronic device 100 may capture an image of at least one object by driving the camera, such as Figure 5 shown.

[0088] refer to Figure 6, the electronic device 100 may display an image obtained by the image sensor on the display as a preview image. After focusing on the object to be captured, the user may perform image capture by touching a capture button 510 displayed on the display. Alternatively, the user may perform image capture by focusing on the object to be photographed and maintaining the focus state for a preset time. In addition, the electronic device 100 may display a captured image 520 on the display unit.

[0089] refer to Figure 7 , the electronic device 100 may recognize at least one text included in the captured image 610, and may determine a priority of the at least one text.

[0090] For example, the electronic device 100 can recognize the text included in the image by using the OCR function. The electronic device 100 can divide the captured image into a plurality of regions and recognize the text by region. At this time, the priority of the plurality of regions can be determined based on whether the shape appearing in each of the plurality of regions is similar to the text and the position of each region. In addition, the text recognized in each of the plurality of regions with priority can have the same priority as its corresponding region. For example, the captured image is divided into a first region R1, a second region R2, and a third region R3, wherein the first region R1 has a first priority, the second region R2 has a third priority, and the third region R3 has a second priority, the first text recognized in the first region R1 can have a first priority, the second text recognized in the second region R2 can have a third priority, and the third text recognized in the third region R3 can have a second priority. However, the embodiment is not limited thereto.

[0091] Alternatively, the electronic device 100 can determine the priority of the recognized text based on whether the shape of the text appearing in the image is similar to the text, the position of the text in the image, the size of the text, the orientation of the text, the color of the text, the font, whether the recognized text is printed text or handwritten text, the language type in the text, the category of the text, etc. However, the embodiment is not limited to this. For example, the electronic device 100 can set the highest priority for the text recognized at the position closest to the center point of the image among the recognized texts. Alternatively, the electronic device 100 can set the highest priority for the text with the largest size among the recognized texts. Alternatively, the electronic device 100 can set the highest priority for the text whose orientation is closest to the horizontal orientation among the recognized texts. However, the embodiment is not limited to this.

[0092] According to an embodiment, the electronic device 100 may include a language model 620 which may generate at least one recommendation word based on at least one text recognized in an image and context information obtained from the electronic device 100 .

[0093] For example, context information may include time information of the electronic device, location information of the electronic device, information about the other party sending and receiving the message (e.g., Robin), the content of the message received from the other party (e.g., where are you?), and the content of the message entered in the message input area (e.g., I am here).

[0094] The language model 620 may be a model that models the language used by humans to calculate the degree to which a specific word string corresponds to the language actually used as a probability. The language model 620 may be a model that predicts grammatically appropriate or highly probable words or letters based on a dictionary, frequency of use, probability of use, etc. For example, in the case of a language model of a character or word unit, the language model may predict the next letter or word based on the letters or words input so far. In addition, in the case of a language model of a sentence unit, the language model may predict the words included in the sentence, taking into account the comprehensive meaning or context of the sentence or paragraph.

[0095] The language model 620 may be a neural network-based model. For example, a model such as a deep neural network (DNN), a recurrent neural network (RNN), or a bidirectional recurrent deep neural network (BRDNN) may be used as the language model 620, but is not limited thereto.

[0096] According to an embodiment, the electronic device 100 may change at least one text (text 1, text 2, ..., text N) recognized in the image into a word matching the context information by using the language model 620. In addition, the electronic device 100 may re-determine the priority of the changed word by using the language model 620.

[0097] For example, based on the content of a message received from another party (e.g., where are you?) and the content of a message entered in the message input area (e.g., "I'm at"), the electronic device 100 can predict that the next word that appears is a word related to the location. Therefore, the electronic device 100 can determine a word that has a high correlation with the location from at least one text as a recommended word, and assign a corresponding priority. In addition, based on the location information of the electronic device 100, the electronic device 100 can determine a word that has a high correlation with the current location information from at least one recognized text (text 1, text 2, ..., text N) as a recommended word, and assign a corresponding priority. However, the embodiments are not limited to this.

[0098] Although not shown, according to an embodiment, the electronic device 100 may generate a recommendation word by changing the type of language in the recognized text based on the language included in the message received from the other party and the language included in the message input into the message input area. For example, when the recognized text is "Garosu-gil" written in English, and the message sent and received between the first user and the second user is written in Korean (e.g., ), the electronic device 100 may generate a text obtained by converting "Garosu-gil" into Korean text. as a recommendation word. However, the embodiment is not limited thereto. ”

[0099] refer to Figure 8 and Fig. 9 , the electronic device 100 can display the recommended words generated using the language model on the display unit.

[0100] For example, Figure 8 As shown, the electronic device 100 may display the recommended words (e.g., "Garosu-gil", "Garosu", "Book store", etc.) at the lower part of the message input area. In addition, according to the priority of the recommended words, the electronic device 100 may display the recommended words in the order of a first recommended word with the highest priority (e.g., "Garosu-gil"), a second recommended word with a second priority (e.g., "Garosu"), and a third recommended word with a third priority (e.g., "Book store"). However, the embodiment is not limited thereto.

[0101] In addition, if Fig. 9 As shown, the electronic device 100 may display the recommendation words by using a pop-up window 710 .

[0102] At this time, when there is a word to be input by the user among the recommended words, the user can select the corresponding word and input the "OK" button. On the other hand, when there is no word to be input by the user among the recommended words, the user can input the "Close" button and terminate the display of the pop-up window 710.

[0103] refer to Fig.10 According to an embodiment, the electronic device 100 may input a word selected from the recommended words into the message input area.

[0104] For example, when receiving the selection Figure 8 When a user input of one of the recommended words displayed in is received, or when a user input of selecting one of the recommended words displayed and selecting Fig. 9When the user inputs the "OK" button in the message input area, the electronic device 100 can input the selected word ("Garosu-gil") into the message input area. In addition, the electronic device 100 can convert the selected word into another language type by using a language model and input the converted word. For example, when the Korean language is included in the message sent and received with the external device and the selected word is "Garosu-gil" written in English, the electronic device can convert "Garosu-gil" written in English into Korean and input it.

[0105] Fig.11 is a block diagram showing a configuration of an electronic device according to another embodiment. Fig.11 The electronic device 300 may be Figure 1 An embodiment of an electronic device 100.

[0106] refer to Fig.11 According to an embodiment, the electronic device 300 may include a processor 330, a sensor 320, a communicator 340, an output unit 350, a user input unit 360, an audio / video (A / V) input unit 370, and a storage unit 380.

[0107] Fig.11 processor 330, Fig.11 The storage unit 380 and Fig.11 The display unit 351 may correspond to Figure 2 processor 120, Figure 2 The memory 130 and Figure 2 The display 140. In reference Fig.11 The description will be omitted Figure 2 Same content as described in .

[0108] The communicator 340 may include one or more components for communicating between the electronic device 300 and an external device (eg, a server). For example, the communicator 340 may include a short-range communication unit 341, a mobile communicator 342, and a broadcast receiving unit 343.

[0109] The short-range communication unit 341 may include a Bluetooth communication unit, a near field communication unit, a wireless LAN (Wi-Fi) communication unit, a Zigbee communication unit, an infrared data association (IrDA) communication unit, a Wi-Fi Direct (WFD) communication unit, an ultra-wideband (UWB) communication unit, and an Ant+ communication unit, but is not limited thereto.

[0110] The mobile communicator 342 transmits and receives wireless signals to and from at least one of a base station, an external terminal or a server on a mobile communication network. Here, the wireless signals may include various types of data according to transmission and reception of voice call signals, video call signals or text / multimedia messages.

[0111] The broadcast receiving unit 343 receives a broadcast signal and / or broadcast related information from the outside through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. Depending on the embodiment, the electronic device 300 may not include the broadcast receiving unit 343.

[0112] The communicator 340 may receive at least one image from an external device. Alternatively, the communicator 340 may request and receive a feature extraction model and a classification model from an external server. The communicator 340 may send the classification results of the plurality of images to the external server, and receive an updated feature extraction model and an updated classification model based on the classification results.

[0113] The output unit 350 is used to output an audio signal, a video signal or a vibration signal, and may include a display unit 351, an audio output unit 352, a vibration motor 353, etc. Fig.11 The display unit 351 is described, and therefore, the description of the display unit 351 will be omitted here.

[0114] The audio output unit 352 outputs audio data received from the communicator 340 or stored in the storage unit 380. In addition, the audio output unit 352 outputs acoustic signals (e.g., call signal reception sound, message reception sound, and notification sound) related to the functions performed by the electronic device 100. The audio output unit 352 may include a speaker, a buzzer, and the like.

[0115] The vibration motor 353 may output a vibration signal. For example, the vibration motor 353 may output a vibration signal corresponding to the output of audio data or video data (e.g., a call signal reception sound, a message reception sound, etc.). In addition, when a touch is input to the touch screen, the vibration motor 353 may output a vibration signal.

[0116] The processor 330 controls the overall operation of the electronic device 300. For example, the processor 330 may control the communicator 340, the output unit 350, the user input unit 360, the sensor 320, and the A / V input unit by executing a program stored in the storage unit 380.

[0117] The user input unit 360 refers to a device for inputting data for controlling the electronic device 300 by a user. For example, the user input unit 360 may include a keyboard, a dome switch, a touch pad (capacitive overlay, pressure-resistant overlay, infrared beam, surface acoustic wave, integral strain gauge, piezoelectric, etc.), a jog wheel, a jog switch, etc., but is not limited thereto.

[0118] The sensor 320 may include not only a sensor that detects biometric information of a user but also a sensor that detects a state of the electronic device 300 or a state around the electronic device 300. In addition, the sensor 320 may transmit information detected by the sensor to the processor 330.

[0119] The sensor 320 may include at least one of a magnetic sensor, an acceleration sensor, a temperature / humidity sensor, an infrared sensor, a gyroscope sensor, a position sensor (e.g., a global positioning system (GPS)), an air pressure sensor, a proximity sensor, and an RGB sensor (illuminance sensor), but is not limited thereto. Those skilled in the art can intuitively infer the function of each sensor from the name, so a detailed description thereof will be omitted here.

[0120] The A / V input unit 370 is used to input an audio signal or a video signal, and may include a camera 371, a microphone 372, etc. The camera 371 may obtain image frames such as still images, videos, etc. through an image sensor in a video call mode or a capture mode. The image captured by the image sensor may be processed by the processor 330 or a separate image processor (not shown).

[0121] The image frame obtained by the camera 371 may be stored in the storage unit 380, or may be transmitted to the outside through the communicator 340. According to the configuration aspect of the electronic device 300, two or more cameras 371 may be provided.

[0122] The microphone 372 receives an external acoustic signal and processes the received external acoustic signal into electrical voice data. For example, the microphone 372 can receive a sound signal from an external device or a speaker. The microphone 372 can use various noise cancellation algorithms to remove the noise generated in the process of receiving the external acoustic signal.

[0123] The storage unit 380 may store a program for processing and controlling the processor 330 , and may store input / output data (eg, applications, contents, time zone information of an external device, an address book, etc.).

[0124] The storage unit 380 may include at least one type of storage medium of the following types: a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., Secure Digital (SD) or eXtreme Digital (XD) memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, or an optical disk. In addition, the electronic device 300 may operate a network storage or cloud server that performs the storage function of the storage unit 380 on the Internet.

[0125] The programs stored in the storage unit 380 may be classified into a plurality of modules according to their functions, such as a user interface (UI) module 381 , a touch screen module 382 , a notification module 383 , and the like.

[0126] The UI module 381 may provide a dedicated UI, a graphical user interface (GUI), etc., which is linked to the electronic device 300 for each application. The touch screen module 382 may detect a touch gesture on the user's touch screen and transmit information about the touch gesture to the processor 330.

[0127] The touch screen module 382 may recognize and analyze a touch code. The touch screen module 382 may be configured with separate hardware including a controller.

[0128] The notification module 383 may generate a signal for notifying an event occurring in the electronic device 300. The event occurring in the electronic device 300 includes call signal reception, message reception, key signal input, schedule notification, etc. The notification module 383 may output the notification signal in the form of a video signal through the display unit 351, or may output the notification signal in the form of an audio signal through the audio output unit 352, or may output the notification signal in the form of a vibration signal through the vibration motor 353.

[0129] at the same time, Figure 2 and Fig.11 The block diagram of the electronic device 100 and 300 shown in the figure is a block diagram of an embodiment. Depending on the specifications of the electronic device 100 and 300 actually implemented, each component of the block diagram may be integrated, added or omitted. That is, if necessary, two or more components may be combined into one component, or one component may be divided into two or more components. In addition, the functions performed in each block are used to describe the embodiment, and the specific operations or devices do not limit the scope of the present disclosure.

[0130] According to an embodiment, the operating method of the electronic device can be implemented in the form of program commands that can be run by various computer devices and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination. The program commands recorded on the medium may be specially designed and configured for the present disclosure, or may be known and available to technicians in the field of computer software. Computer-readable recording media include, for example, magnetic media, such as hard disks, floppy disks, and tapes; optical media, such as compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs); magneto-optical media, such as floppy disks, and hardware devices specially configured to store and run program commands, such as magneto-optical media, as well as ROMs, RAMs, flash memories, etc. Program commands, for example, include not only machine language codes made by compilers, but also high-level language codes that can be run by computers using interpreters, etc.

[0131] Although the embodiments have been described in detail above, the scope of the present disclosure is not limited thereto, and various modifications and improvements made by those skilled in the art based on the basic concept of the present disclosure defined in the appended claims also fall within the scope of the present disclosure.

Claims

1. An electronic device, comprising: a camera configured to capture an image of at least one object; monitor; a memory storing one or more instructions; as well as A processor configured to execute one or more instructions stored in the memory to: recognizing at least one text included in an image received from a camera, generating one or more recommendation words with priorities based on the recognized at least one text and context information, and controlling a display to display the generated one or more recommendation words according to the priorities, The context information includes at least one of time information of the electronic device, location information of the electronic device, content of a message received from an external device, information about the external device, or content of a message input into a message input area.

2. The electronic device according to claim 1, wherein: The processor is further configured to execute one or more instructions to: When the electronic device is operated in a text input mode, controlling the display to display a text input window and a capture icon; as well as When a user input selecting a capture icon is received, the camera is driven and controlled to capture an image of the at least one object.

3. The electronic device according to claim 1, wherein: The processor is also configured to execute one or more instructions to change the recognized at least one text into a word matching the context information by using a language model.

4. The electronic device according to claim 1, wherein: The processor is also configured to execute one or more instructions to determine a priority of the identified at least one text based on at least one of a location of the at least one text in the image, a size of the at least one text, a color of the at least one text, or a type of the at least one text.

5. The electronic device according to claim 1, wherein: The processor is further configured to execute one or more instructions to determine a priority of the one or more recommended words based on the recognized category of at least one text and context information of the text input to the electronic device.

6. The electronic device according to claim 1, wherein: The electronic device is in a state of sending and receiving messages with the external device.

7. The electronic device according to claim 1, wherein: The processor is further configured to execute one or more instructions to control the display to display the one or more recommended words on the display in order starting from the highest priority.

8. The electronic device according to claim 1, wherein: The processor is further configured to execute one or more instructions to control inputting the selected recommended word into the text input window upon receiving a user input selecting any one of the one or more recommended words.

9. An operating method of an electronic device, the operating method comprising: obtaining an image by capturing an image of at least one object; identifying at least one text included in the image; generating one or more recommendation words with priorities based on the identified at least one text and context information; as well as displaying the generated one or more recommendations according to priority, and The context information includes at least one of time information of the electronic device, location information of the electronic device, content of a message received from an external device, information about the external device, or content of a message input into a message input area.

10. The operating method according to claim 9, further comprising: When the electronic device operates in a text input mode, a text input window and a capture icon are displayed, Wherein, obtaining the image by capturing the image of at least one object includes: when a user input selecting a capture icon is received, driving the camera and capturing the image of the at least one object.

11. The operating method according to claim 9, wherein: Generating one or more recommended words with priorities includes: changing the recognized at least one text into words matching the context information by using a language model.

12. The operating method according to claim 9, wherein: Identifying at least one text included in the image includes determining a priority of the identified at least one text based on at least one of a position of the at least one text in the image, a size of the at least one text, a color of the at least one text, or a type of the at least one text.

13. The operating method according to claim 9, wherein: Generating one or more recommended words with priorities includes determining the priorities of the one or more recommended words based on the recognized category of the at least one text and context information of the text input to the electronic device.

14. The operating method according to claim 9, wherein: The electronic device is in a state of sending and receiving messages with the external device.

15. A computer-readable recording medium having recorded thereon a program for executing the operating method according to claim 9 on a computer.

Images