Electronic device and method for recognizing sentences represented by strokes

Abstract

A processor of an electronic device may be configured to, in response to a first input indicating that at least one character distinguished by a plurality of strokes is selected, display a first visual object for selecting a first sentence including the at least one character. The processor may be configured to, in response to a second input indicating that the first visual object is selected, identify a stroke included in the first sentence from a plurality of strokes based on a pitch between a first word and a second word including at least one character and a point in time at which a plurality of words including the first word and the second word are input. The processor may be configured to display a second visual object corresponding to an identified stroke of the plurality of strokes.

Description

Technical Field

[0001] One or more embodiments relate to an electronic device and method for recognizing a sentence indicated by strokes. Background Art

[0002] An electronic device is being developed for extracting text from handwriting indicated by strokes drawn by a user. For example, a user may draw strokes indicating handwriting by moving a finger, stylus, and / or digitizer in contact with a display of an electronic device, or by moving a pointing device (e.g., a mouse) connected to the electronic device. The electronic device may recognize characters (e.g., letters) based on the strokes. Based on recognizing one or more characters, the electronic device may recognize text including the one or more characters. Summary of the invention

[0003] Technical Solution

[0004] According to an embodiment, an electronic device may include a display and a processor. The processor may be configured to display a plurality of strokes in the display. The processor may be configured to display a first visual object for identifying a first sentence including at least one character in response to a first input indicating the selection of at least one character distinguished by a plurality of strokes. The processor may be configured to identify the strokes included in the first sentence among the plurality of strokes based on the spacing between the first word and the second word including at least one character and the moment when a plurality of words including the first word and the second word are input in response to a second input indicating the selection of the first visual object. The processor may be configured to display a second visual object corresponding to the recognized strokes among the plurality of strokes in the display based on the recognition of the strokes included in the first sentence.

[0005] According to an embodiment, a method of an electronic device may include displaying a plurality of characters in a display of the electronic device. The method may include displaying a list of functions associated with at least one first character in response to a first input indicating that at least one first character is selected from a plurality of characters. The method may include identifying a sentence including at least one first character from a plurality of characters in response to a second input indicating that a first function for identifying a sentence including at least one first character is selected based on list recognition. The method may include displaying at least one visual object associated with a plurality of second characters included in the sentence and including at least one first character based on the recognition of the sentence.

[0006] According to an embodiment, an electronic device may include a display and a processor. The processor may be configured to display a plurality of characters in the display. The processor may be configured to display a function list associated with at least one first character in response to a first input indicating that at least one first character is selected from a plurality of characters. The processor may be configured to identify a sentence including at least one first character of a plurality of characters in response to a second input indicating that a first function for identifying a sentence including at least one first character is selected based on list recognition. The processor may be configured to display at least one visual object associated with a plurality of second characters included in the sentence and including at least one first character based on the recognition of the sentence.

[0007] According to an embodiment, a method of an electronic device may include displaying a plurality of strokes in a display of the electronic device. The method may include displaying a first visual object for identifying a first sentence including at least one character in response to a first input indicating selection of at least one character distinguished by a plurality of strokes. The method may include identifying a stroke included in the first sentence among a plurality of strokes based on a spacing between a first word including at least one character and a second word and a moment when a plurality of words including the first word and the second word are input in response to a second input indicating selection of the first visual object. The method may include displaying a second visual object corresponding to an identified stroke among a plurality of strokes in a display based on identification of the strokes included in the first sentence. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a block diagram of an electronic device according to an embodiment.

[0009] Figure 2 An example in which an electronic device displays a plurality of strokes for indicating handwriting according to an embodiment is shown.

[0010] Figure 3 An example of one or more programs executed by an electronic device to recognize at least one sentence from a plurality of strokes according to an embodiment is shown.

[0011] Figure 4 An example of an operation of an electronic device obtaining media content including at least one stroke based on interaction between a user and the electronic device according to an embodiment is shown.

[0012] Figure 5 Examples of words, characters and / or sentences recognized by an electronic device from a plurality of strokes according to an embodiment are shown.

[0013] Figure 6 An example of an operation of an electronic device recognizing a distance between words distinguished by a plurality of strokes according to an embodiment is shown.

[0014] Figure 7An example of an operation of an electronic device recognizing a sentence from a plurality of strokes according to an embodiment is shown.

[0015] Figures 8a to 8c An example of an operation of an electronic device recognizing a word, a character, and / or a sentence based on a user's gesture according to an embodiment is shown.

[0016] Figure 9a to Figure 9b An example of an operation of an electronic device recognizing a sentence from media content including a plurality of strokes according to an embodiment is shown.

[0017] Fig.10 An example of an operation of an electronic device according to an embodiment to recognize a sentence from media content including a plurality of strokes and an image related to the sentence in the media content is shown.

[0018] Fig.11 An example of a flowchart for describing an operation of obtaining one or more sentences from a plurality of strokes by an electronic device according to an embodiment is shown.

[0019] Figure 12a to Figure 12b An example of a flowchart for describing an operation performed by an electronic device based on an input received from a user according to an embodiment is shown.

[0020] Fig.13 An example of a flowchart for describing an operation performed by an electronic device based on an input received from a user according to an embodiment is shown. DETAILED DESCRIPTION

[0021] Hereinafter, various embodiments of the present document will be described with reference to the accompanying drawings.

[0022] The various embodiments and terms used herein are not intended to limit the technical features described herein to specific embodiments, but should be understood to include various modifications, equivalents and / or substitutions of the embodiments. With respect to the description of the figures, similar reference numerals may be used for similar or related components. The nouns in the singular form corresponding to the term may include one or more things, unless otherwise clearly indicated by the relevant context. In this document, each of the phrases such as "A or B", "at least one of A and B", "at least one of A, B and C", "at least one of A, B or C" and "at least one of A, B or C" may include any one of the phrases, or all possible combinations thereof. Terms such as "first", "second" or "first" or "second" may be used only to distinguish a corresponding component from another corresponding component, and are not limited to other aspects (e.g., importance or order). When a certain (e.g., first) component is referred to as "connected" or "connected" to another (e.g., second) component, whether or not it carries the term "functional" or "communicationally", it means that some components may be directly (e.g., wired) connected, wirelessly connected or connected through a third component.

[0023] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and is used interchangeably with terms such as logic, logic block, component, or circuit. A module may be a minimum unit or a portion of an integrated configured component or a component that performs one or more functions. For example, according to an embodiment, a module may be implemented in the form of an application specific integrated circuit (ASIC).

[0024] Figure 1 1 is a block diagram illustrating an electronic device 101 in a network environment 100 according to various embodiments. Figure 1 , the electronic device 101 in the network environment 100 may communicate with the electronic device 102 via the first network 198 (e.g., a short-range wireless communication network), or communicate with at least one of the electronic device 104 or the server 108 via the second network 199 (e.g., a long-range wireless communication network). According to an embodiment, the electronic device 101 may communicate with the electronic device 104 via the server 108. According to an embodiment, the electronic device 101 may include a processor 120, a memory 130, an input module 150, a sound output module 155, a display module 160, an audio module 170, a sensor module 176, an interface 177, a connection terminal 178, a haptic module 179, a camera module 180, a power management module 188, a battery 189, a communication module 190, a user identification module (SIM) 196, or an antenna module 197. In some embodiments, at least one of the components (e.g., the connection terminal 178) may be omitted from the electronic device 101, or one or more other components may be added to the electronic device 101. In some embodiments, some of the components (eg, sensor module 176 , camera module 180 , or antenna module 197 ) may be implemented as a single component (eg, display module 160 ).

[0025] The processor 120 may execute, for example, software (e.g., program 140) to control at least one other component (e.g., hardware or software component) of the electronic device 101 that is coupled to the processor 120, and may perform various data processing or calculations. According to an embodiment, as at least a part of the data processing or calculation, the processor 120 may store a command or data received from another component (e.g., sensor module 176 or communication module 190) in the volatile memory 132, process the command or data stored in the volatile memory 132, and store the resulting data in the non-volatile memory 134. According to an embodiment, the processor 120 may include a main processor 121 (e.g., a central processing unit (CPU) or an application processor (AP)), or an auxiliary processor 123 (e.g., a graphics processing unit (GPU), a neural processing unit (NPU), an image signal processor (ISP), a sensor hub processor, or a communication processor (CP)) that may operate independently of the main processor 121 or in conjunction with it. For example, when the electronic device 101 includes a main processor 121 and an auxiliary processor 123, the auxiliary processor 123 may be adapted to consume lower power than the main processor 121 or be designated for a designated function. The auxiliary processor 123 may be implemented separately from the main processor 121 or as part of the main processor.

[0026] When the main processor 121 is in an inactive (e.g., sleep) state, in place of the main processor 121, or when the main processor 121 is in an active state (e.g., executing an application), together with the main processor 121, the auxiliary processor 123 may control at least some of the functions or states related to at least one component (e.g., display module 160, sensor module 176, or communication module 190) among the components of the electronic device 101. According to an embodiment, the auxiliary processor 123 (e.g., image signal processor or communication processor) may be implemented as part of another component (e.g., camera module 180 or communication module 190) that is functionally related to the auxiliary processor 123. According to an embodiment, the auxiliary processor 123 (e.g., neural processing unit) may include a hardware structure designated for artificial intelligence model processing. The artificial intelligence model may be generated by machine learning. Such learning may be performed, for example, by the electronic device 101 that performs artificial intelligence or via a separate server (e.g., server 108). The learning algorithm may include, but is not limited to, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. The artificial intelligence model may include a plurality of artificial neural network layers. The artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q network, or a combination of two or more thereof, but is not limited thereto. In addition to the hardware structure, the artificial intelligence model may additionally or alternatively include a software structure.

[0027] The memory 130 may store various data used by at least one component of the electronic device 101 (e.g., the processor 120 or the sensor module 176). The various data may include, for example, input data or output data of software (e.g., the program 140) and commands related thereto. The memory 130 may include a volatile memory 132 or a non-volatile memory 134.

[0028] The program 140 may be stored as software in the memory 130 , and may include, for example, an operating system (OS) 142 , middleware 144 , or application programs 146 .

[0029] The input module 150 may receive a command or data to be used by another component (e.g., the processor 120) of the electronic device 101 from outside (e.g., a user) of the electronic device 101. The input module 150 may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0030] The sound output module 155 can output sound signals to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as playing multimedia or playing recordings. The receiver may be used to receive incoming calls. Depending on the embodiment, the receiver may be implemented as a part of the speaker or as a part of the speaker.

[0031] The display module 160 can visually provide information to the outside of the electronic device 101 (e.g., a user). The display module 160 may include, for example, a display, a holographic device, or a projector, and a control circuit for controlling a corresponding one of the display, the holographic device, and the projector. According to an embodiment, the display module 160 may include a touch sensor adapted to detect a touch, or a pressure sensor adapted to measure the strength of a force caused by a touch.

[0032] The audio module 170 may convert sound into an electrical signal, and vice versa. According to an embodiment, the audio module 170 may obtain sound via the input module 150, or output sound via the sound output module 155 or an earphone of an external electronic device (e.g., electronic device 102) directly (e.g., wired) or wirelessly connected to the electronic device 101.

[0033] The sensor module 176 may detect an operating state (e.g., power or temperature) of the electronic device 101 or an environmental state (e.g., a user's state) outside the electronic device 101, and then generate an electrical signal or data value corresponding to the detected state. According to an embodiment, the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, an atmospheric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illumination sensor.

[0034] The interface 177 may support one or more designated protocols for directly (e.g., wiredly) or wirelessly connecting the electronic device 101 with an external electronic device (e.g., the electronic device 102). According to an embodiment, the interface 177 may include, for example, a high-definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface, or an audio interface.

[0035] The connection end 178 may include a connector, via which the electronic device 101 may be physically connected to an external electronic device (e.g., the electronic device 102). According to an embodiment, the connection end 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0036] The haptic module 179 may convert the electrical signal into mechanical stimulation (eg, vibration or motion) or electrical stimulation, which may be recognized by the user via his sense of touch or kinesthetic sense. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0037] The camera module 180 may capture still images or moving images. According to an embodiment, the camera module 180 may include one or more lenses, an image sensor, an image signal processor, or a flash.

[0038] The power management module 188 may manage power supplied to the electronic device 101. According to an embodiment, the power management module 188 may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0039] The battery 189 may supply power to at least one component of the electronic device 101. According to an embodiment, the battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0040] The communication module 190 may support establishing a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (e.g., electronic device 102, electronic device 104, or server 108), and perform communication via the established communication channel. The communication module 190 may include one or more communication processors capable of operating independently from the processor 120 (e.g., an application processor (AP)), and support direct (e.g., wired) communication or wireless communication. According to an embodiment, the communication module 190 may include a wireless communication module 192 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module), or a wired communication module 194 (e.g., a local area network (LAN) communication module or a power line communication (PLC) module). A corresponding one of these communication modules can communicate with an external electronic device via a first network 198 (e.g., a short-range communication network such as Bluetooth™, Wireless Fidelity (Wi-Fi) Direct, or Infrared Data Association (IrDA)) or a second network 199 (e.g., a long-range communication network such as a traditional cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These various types of communication modules can be implemented as a single component (e.g., a single chip), or can be implemented as multiple components separated from each other (e.g., multiple chips). The wireless communication module 192 can use the subscriber information (e.g., International Mobile Subscriber Identity (IMSI)) stored in the subscriber identification module 196 to identify and authenticate the electronic device 101 in the communication network (such as the first network 198 or the second network 199).

[0041] The wireless communication module 192 can support 5G networks and next-generation communication technologies after the 4G network, such as new radio (NR) access technology. NR access technology can support enhanced mobile broadband (eMBB), massive machine type communication (mMTC), or ultra-reliable low-latency communication (URLLC). The wireless communication module 192 can support high-frequency bands (e.g., millimeter wave bands) to achieve, for example, high data transmission rates. The wireless communication module 192 can support various technologies for ensuring performance on high-frequency bands, such as, for example, beamforming, massive multiple-input multiple-output (massive MIMO), full-dimensional MIMO (FD-MIMO), array antennas, analog beamforming, or massive antennas. The wireless communication module 192 can support various requirements specified in the electronic device 101, an external electronic device (e.g., electronic device 104), or a network system (e.g., a second network 199). According to an embodiment, the wireless communication module 192 may support: peak data rates for implementing eMBB (e.g., 20 Gbps or higher), loss coverage for implementing mMTC (e.g., 164 dB or less), or U-plane latency for implementing URLLC (e.g., 0.5 ms or less, or 1 ms or less round trip for each of the downlink (DL) and uplink (UL)).

[0042] The antenna module 197 may transmit a signal or power to the outside of the electronic device 101 (e.g., an external electronic device) or receive a signal or power from the outside. According to an embodiment, the antenna module 197 may include an antenna including a radiation element composed of a conductive material or a conductive pattern formed in or on a substrate (e.g., a printed circuit board (PCB)). According to an embodiment, the antenna module 197 may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication scheme used in a communication network (such as the first network 198 or the second network 199) may be selected from the plurality of antennas by, for example, the communication module 190 (e.g., the wireless communication module 192). Then, via the selected at least one antenna, a signal or power is transmitted or received between the communication module 190 and the external electronic device. According to an embodiment, another component (e.g., a radio frequency integrated circuit (RFIC)) other than the radiation element may be additionally formed as a part of the antenna module 197.

[0043] According to various embodiments, the antenna module 197 may form a millimeter wave antenna module. According to an embodiment, the millimeter wave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., millimeter wave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving signals of the specified high frequency band.

[0044] At least some of the above components may be interconnected and transmit signals (eg, commands or data) therebetween via an inter-peripheral communication scheme (eg, a bus, a general purpose input output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI)).

[0045] According to an embodiment, a command or data may be transmitted or received between the electronic device 101 and the external electronic device 104 via a server 108 connected to the second network 199. Each of the electronic devices 102 or 104 may be a device of the same type or a different type as the electronic device 101. According to an embodiment, all or some of the operations to be performed at the electronic device 101 may be performed at one or more of the external electronic devices 102, 104 or 108. For example, if the electronic device 101 should automatically perform a function or service, or perform a function or service in response to a request from a user or another device, the electronic device 101 may request one or more external electronic devices to perform at least a portion of the function or service instead of or in addition to performing the function or service. The one or more external electronic devices that receive the request may perform at least a portion of the requested function or service, or perform another function or another service related to the request, and transmit the result of the execution to the electronic device 101. The electronic device 101 may provide the result as at least a part of the response to the request in the case of further processing the result or in the case of not further processing the result. To this end, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device 101 may use, for example, distributed computing or mobile edge computing to provide ultra-low latency services. In another embodiment, the external electronic device 104 may include an Internet of Things (IoT) device. The server 108 may be an intelligent server using machine learning and / or neural networks. According to an embodiment, the external electronic device 104 or the server 108 may be included in the second network 199. The electronic device 101 may be applied to smart services (e.g., smart homes, smart cities, smart cars, or healthcare) based on 5G communication technology or IoT-related technology.

[0046] The electronic device according to various embodiments may be one of various types of electronic devices. The electronic device may include, for example, a portable communication device (e.g., a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance. According to an embodiment of the present disclosure, the electronic device is not limited to those devices described above.

[0047] It should be understood that the various embodiments of the present disclosure and the terms used therein are not intended to limit the technical features set forth herein to specific embodiments, but include various changes, equivalent forms or alternative forms for the corresponding embodiments. With regard to the description of the figures, similar reference numerals may be used to refer to similar or related elements. It will be understood that the nouns in the singular form corresponding to the term may include one or more things unless the relevant context clearly indicates otherwise. As used herein, each of the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C" and "at least one of A, B or C" may include any one or all possible combinations of the terms enumerated together in the corresponding one of the phrases. As used herein, terms such as "1st" and "2nd" or "first" and "second" may be used to simply distinguish the corresponding component from another component, and do not limit the component in other aspects (e.g., importance or order). It will be understood that if an element (e.g., a first element) is referred to as being “coupled with,” “coupled to,” “connected with,” or “connected to” another element (e.g., a second element), with or without the term “operably” or “communicatively,” it means that the element can be connected to the other element directly (e.g., wired), wirelessly, or via a third element.

[0048] As used in conjunction with various embodiments of the present disclosure, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with other terms (e.g., "logic," "logic block," "portion," or "circuit"). A module may be a single integrated component adapted to perform one or more functions, or its smallest unit or portion. For example, depending on the embodiment, a module may be implemented in the form of an application specific integrated circuit (ASIC).

[0049] Various embodiments as set forth herein may be implemented as software (e.g., program 140) that includes one or more instructions stored in a storage medium (e.g., internal memory 136 or external memory 138) that can be read by a machine (e.g., electronic device 101). For example, a processor (e.g., processor 120) of a machine (e.g., electronic device 101) may call at least one of the one or more instructions stored in the storage medium, and execute the instructions with or without one or more other components under the control of the processor. This allows the machine to be operated to perform at least one function according to at least one instruction called. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Among them, the term "non-transitory" simply means that the storage medium is a tangible device and does not include signals (e.g., electromagnetic waves), but this term does not distinguish between data being semi-permanently stored in the storage medium and data being temporarily stored in the storage medium.

[0050] According to an embodiment, the method according to various embodiments of the present disclosure may include and be provided in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)), or distributed online via an application store (e.g., PlayStore™) or distributed directly between two user devices (e.g., smart phones) (e.g., downloaded or uploaded). If distributed online, at least a portion of the computer program product may be temporarily generated or at least temporarily stored in a machine-readable storage medium (e.g., a memory of a manufacturer's server, an application store's server, or a relay server).

[0051] According to various embodiments, each of the above-mentioned components (e.g., a module or a program) may include a single entity or multiple entities, and some of the multiple entities may be separately arranged in different components. According to various embodiments, one or more of the above-mentioned components may be omitted, or one or more other components may be added. Alternatively or additionally, multiple components (e.g., a module or a program) may be integrated into a single component. In this case, according to various embodiments, the integrated component may still perform one or more functions of each of the multiple components in the same or similar manner as a corresponding one of the multiple components performing one or more functions before integration. According to various embodiments, the operations performed by a module, a program or another component may be performed sequentially, in parallel, repeatedly or in a heuristic manner, or one or more of the operations may be run or omitted in a different order, or one or more other operations may be added.

[0052] Figure 2 An example of an electronic device displaying a plurality of strokes for indicating handwriting according to an embodiment is shown. Figure 2 The electronic device 101 may be a terminal owned by a user. For example, the terminal may include a personal computer (PC), a smart phone, a smart tablet, a tablet computer, a smart watch, and a smart accessory such as a head mounted device (HMD).

[0053] According to an embodiment, the electronic device 101 may include a display 210. According to an embodiment, the electronic device 101 may visualize handwriting by using the display 210. Figure 3 An example of hardware and / or software included in electronic device 101 for visualizing handwriting is described. Handwriting may include one or more lines drawn by a user on a surface to convey linguistic meaning. Hereinafter, one or more lines included in handwriting may be referred to as strokes.

[0054] According to an embodiment, the electronic device 101 may obtain information about multiple strokes included in the handwriting by digitally converting the simulated handwriting. The information may include the position and / or order of each of the multiple strokes. For example, the electronic device 101 may receive input indicating multiple strokes based on a pointing device (such as a fingertip, a stylus, a digitizer, and / or a mouse for adjusting the cursor position) in contact with the display 210. For example, a stroke may be a trajectory and / or path dragged by an input tool (such as a fingertip, a stylus, and / or a digitizer) when the input tool is in contact with the display 210. For example, a stroke may be a trajectory and / or path of a cursor and / or a mouse pointer of a pointing device connected to the electronic device 101 in the display 210. According to an embodiment, reference will be made to Figure 4 The operation of the electronic device 101 obtaining information and media content including the information based on a gesture of a user drawing a plurality of strokes is described.

[0055] In an embodiment, media content may include information generated based on the interaction between the user and the electronic device 101. The media content may include information for reconstructing handwriting, text, images, videos, and / or audio received from the user. In an embodiment, the media content may be referred to as a document and / or a memo. For example, information indicating one or more strokes drawn by the user may be included in the media content. For example, images, videos, and / or audio embedded by the user may be included in the media content. In a state of reconstructing media content including audio, the electronic device 101 may display a visual object, such as an icon 250, for reproducing the audio in the media content. Reconstructing the media content may include the electronic device 101 visualizing the information included in the media content within the display 210.

[0056] refer to Figure 2, according to an embodiment, an example of a state in which the electronic device 101 visualizes media content in the display 210 is shown. The electronic device 101 may display at least a portion of the media content in the area 220 of the display 210. The electronic device 101 may display a name and / or title assigned to the media content in the area 230 of the display 210. The electronic device 101 may display the strokes included in the media content in the area 220. For example, the electronic device 101 may display the strokes in the area 220, including information included in the media content and indicating the moment and / or position of inputting each stroke. The moment of the stroke corresponds to an instance of the time when the stroke is formed on the electronic device 101. Each stroke has a discernible moment that identifies the corresponding time (e.g., a timestamp) when the corresponding stroke is input to the electronic device 101. In addition, the moment of the stroke indicates the corresponding time 101 when the user inputs the stroke to the electronic device. In one or more embodiments, the moment of inputting the stroke may be sorted in chronological order.

[0057] According to an embodiment, the electronic device 101 may recognize characters, words, and / or sentences 240 distinguished by strokes. In an embodiment, the characters recognized by the electronic device 101 may include letters, numbers, and / or symbols distinguished by one or more strokes. In an embodiment, a word (or vocabulary) may include a group of one or more characters having meaning. Words may be distinguished by spacing. In an embodiment, a sentence may include a group of one or more words indicating meaningful content. Sentences may be distinguished by punctuation marks (e.g., periods, exclamation marks, and / or question marks). According to an embodiment, reference may be made to the text of the text. Figure 5 Description of electronic device 101 Figure 2 Operations of identifying sentences, words and / or phrases in media content are shown.

[0058] refer to Figure 2 According to an embodiment, sentences 241, 242, 243, 244, 245, 246, and 247 are shown as examples of sentences 240 recognized by the electronic device 101. According to an embodiment, the electronic device 101 can recognize words distinguished by a plurality of strokes included in the media content to recognize the sentences 241, 242, 243, 244, 245, 246, and 247 from the media content. The electronic device 101 can recognize the sentences 241, 242, 243, 244, 245, 246, and 247 from the media content using the positions of the words and / or the moments when the words are input. According to an embodiment, the reference Figure 6 to Figure 7 The operation of the electronic device 101 to recognize the sentences 241, 242, 243, 244, 245, 246, and 247 based on the location and / or time is described.

[0059] According to an embodiment, in a state where at least a portion of the media content is displayed in area 220, the electronic device 101 may receive an input from a user indicating the selection of a character, a word, and / or a sentence. For example, in response to a first input indicating the selection of a character, the electronic device 101 may identify or select at least one character including at least one stroke identified by the first input. The electronic device 101 may display in the display 210 a visual object indicating that at least one character has been selected by the first input. For example, in response to a second input indicating the selection of a word and / or a row, the electronic device 101 may identify at least one word including at least one stroke selected by the second input, or display the row where at least one word is located. The electronic device 101 may display in the display 210 at least one word identified by the second input and / or a visual object for highlighting the identified row.

[0060] According to an embodiment, in response to a third input indicating selection of a sentence, the electronic device 101 may identify a sentence including at least one stroke selected by the third input. The electronic device 101 may display a visual object for highlighting the sentence identified by the third input in the display 210. Figure 2 , based on receiving the third input at the point where the word "UP" in the sentence 244 is displayed on the display 210, the electronic device 101 may display a visual object for highlighting the sentence 244 instead of the other sentences 241, 242, 243, 245, 246, and 247. According to an embodiment, the electronic device 101 may identify the sentence 244 corresponding to the third input based on the position of the stroke in the media content and / or the moment when the stroke is input. According to an embodiment, the reference Figures 8a to 8c , Figure 9a to Figure 9b and / or Fig.10 To describe the operation of the electronic device 101 to recognize characters, words and / or sentences based on input received from the user.

[0061] As described above, according to an embodiment, in a state of visualizing handwriting based on multiple strokes, the electronic device 101 may receive an input for selecting a character, a word, and / or a sentence distinguished by multiple strokes. For example, in response to an input indicating the selection of a sentence, the electronic device 101 may obtain the spacing between words distinguished by multiple strokes and the time of inputting the word. Based on the spacing and the time, the electronic device 101 may identify a sentence including at least one stroke selected by the input from multiple strokes. The electronic device 101 may highlight the identified sentence from the sentences distinguished by multiple strokes to be more prominent than another sentence different from the identified sentence. By highlighting the identified sentence, the electronic device 101 may indicate that the identified sentence is selected based on the input.

[0062] In the following, reference will be made to Figure 3An example of one or more hardware included in the electronic device 101 according to the embodiment is described.

[0063] Figure 3 An example of one or more programs executed by the electronic device 101 to recognize at least one sentence from a plurality of strokes according to an embodiment is shown. Figure 3 The electronic device 101 may be Figure 1 to Figure 2 An example of an electronic device 101. For example, Figure 2 The electronic device 101 and the display 210 may include Figure 3 An electronic device 101 and a display 210.

[0064] refer to Figure 3 According to an embodiment, the electronic device 101 may include at least one of a processor 120, a memory 130, and a display 210. The processor 120, the memory 130, and the display 210 may be electronically and / or operably connected to each other through electronic components such as a communication bus 305. Hereinafter, operably connected hardware may refer to a direct connection or an indirect connection between hardware established in a wired or wireless manner to ensure that the first hardware is controlled by the second hardware. Although described based on different blocks, the embodiments are not limited thereto, and Figure 3 Part of the hardware shown in (eg, processor 120 and at least a portion of memory 130) may be included in a single integrated circuit such as a system on a chip (SoC). The type and / or number of hardware included in electronic device 101 is not limited to Figure 3 For example, the electronic device 101 may only include Figure 3 A portion of the hardware is shown.

[0065] According to an embodiment, the processor 120 of the electronic device 101 may include a circuit for processing data based on one or more instructions. For example, the hardware components for processing data may include an arithmetic and logic unit (ALU), a floating point unit (FPU), a field programmable gate array (FPGA), and / or a central processing unit (CPU). The number of processors 120 may be one or more. For example, the processor 120 may have a multi-core processor structure, such as dual-core, quad-core, or hexa-core. Figure 3 The processor 120 may include Figure 1 The processor 120.

[0066] According to an embodiment, the memory 130 of the electronic device 101 may include hardware for storing data and / or instructions input to the processor 120 and / or output from the processor. For example, the memory 130 may include a volatile memory (e.g., a random access memory (RAM)) and / or a non-volatile memory (e.g., a read-only memory (ROM)). For example, the volatile memory may include at least one of a dynamic RAM (DRAM), a static RAM (SRAM), a cache RAM, or a pseudo SRAM (PSRAM). For example, the non-volatile memory may include at least one of a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), a flash memory, a hard disk, an optical disk, or an embedded multimedia card (eMMC). Figure 3 The memory 130 may include Figure 1 Memory 130.

[0067] According to an embodiment, the display 210 of the electronic device 101 may output visual information (e.g., Figure 4 , Figures 8a to 8c , Figure 9a to Figure 9b and / or Fig.10 ). For example, the display 210 may be controlled by the processor 120 and output visual information to the user. The display 210 may include a flat panel display (FPD) and / or electronic paper. The FPD may include a liquid crystal display (LCD), a plasma display panel (PDP), and / or one or more light-emitting LEDs. The LED may include an organic LED (OLED).

[0068] According to an embodiment, the display 210 of the electronic device 101 may include a sensor (e.g., a touch sensor panel (TSP)) for detecting an external object (e.g., a user's finger) on the display 210. For example, the electronic device 101 may detect an external object that is in contact with or suspended above the display 210 based on the TSP. In response to detecting the external object, the electronic device 101 may perform a function related to a specific visual object among the visual objects displayed in the display 210 that corresponds to the position of the external object on the display 210.

[0069] According to an embodiment, one or more instructions indicating calculations and / or operations performed on data by the processor 120 may be stored in the memory 130. A collection of one or more instructions may be referred to as firmware, an operating system, a process, a routine, a subroutine, and / or an application. For example, when executing a set of multiple instructions deployed in the form of an operating system, firmware, a driver, and / or an application, the electronic device 101 and / or the processor 120 may perform Figures 11 to 13Hereinafter, the fact that an application is installed in the electronic device 101 may indicate that one or more instructions provided in the form of an application are stored in the memory 130 of the electronic device 101, and means that one or more applications are stored in an executable format (e.g., a file with an extension specified by the operating system of the electronic device 101) by the processor 120 of the electronic device 101.

[0070] refer to Figure 3 , according to an embodiment, a note application 320 , a handwriting recognizer 330 , and a sentence selector 340 are illustrated as examples of application programs stored in the memory 130 . Figure 3 The note application 320 may be application software for displaying a user interface (UI) for performing handwriting-related functions in the display 210 . Figure 3 The handwriting recognizer 330 and / or the sentence selector 340 may be software for obtaining text from one or more strokes indicating handwriting. Although described based on different blocks, the embodiment is not limited thereto. For example, the handwriting recognizer 330 and the sentence selector 340 may be included in the note application 320. The handwriting recognizer 330 and the sentence selector 340 may be executed in a second state different from the first state displayed to the user, such as a background process and / or a daemon process.

[0071] According to an embodiment, the processor 120 in the electronic device 101 may display media content including one or more strokes representing handwriting based on the execution of the note application 320. For example, the electronic device 101 may display the area 220 based on the execution of the note application 320, wherein the media content including one or more strokes representing handwriting is displayed. Figure 2 According to an embodiment, the electronic device 101 may receive an input for executing a function related to handwriting recognition from a user by using the note application 320.

[0072] In an embodiment, handwriting recognition may include a function of recognizing text including characters, words and / or sentences from one or more strokes. Based on handwriting recognition, one or more strokes drawn by a user may be converted by the electronic device 101 into readable information. For example, text recognized from one or more strokes by the electronic device 101 based on handwriting recognition may include information encoded based on multiple bits, such as Unicode and / or American Standard Code for Information Interchange (ASCII) codes. According to an embodiment, the electronic device 101 may execute a handwriting recognizer 330 and / or a sentence selector 340 based on input received through the note application 320 and indicating extraction of text from strokes. For example, the handwriting recognizer 330 and / or the sentence selector 340 may be executed based on input indicating execution of a function such as handwriting recognition.

[0073] According to an embodiment, the electronic device 101 may control the handwriting recognition-related operations of the electronic device 101 based on the execution of the handwriting recognizer 330. For example, the electronic device 101 may call the functions and / or events supported by the handwriting recognizer 330 based on the input received from the user through the note application 320. For example, the functions supported by the handwriting recognizer 330 may include a function of recognizing multiple words by grouping multiple strokes included in the media content. The function of recognizing multiple words may include storing information corresponding to each of the multiple words in the memory 130 based on a data structure such as a connection list. For example, the connection list to which the nodes corresponding to each word distinguished or divided by different strokes are connected may be referred to as a word list. The function of recognizing multiple words may include storing a list of words distinguished by the direction and / or row in which the multiple words are arranged in the memory 130. The list of words distinguished by rows may be referred to as a row list. The function of recognizing multiple words may include storing a list of moments when multiple words are input in the memory 130. The moment list may be referred to as a time list.

[0074] According to an embodiment, the electronic device 101 may identify one or more sentences included in the media content by using the sentence selector 340. The electronic device 101 may identify one or more sentences based on the execution of the sentence selector 340, and the electronic device 101 may identify one or more sentences based on the word list, line list and / or time list stored in the memory 130 by executing the handwriting recognizer 330. For example, the electronic device 101 may obtain spatial information of one or more words in the media content based on the word list and / or line list by using the sentence selector 340. For example, the electronic device 101 may obtain time information in the media content based on the time list by using the sentence selector 340, and the time information includes the moment when one or more words are input and / or the order in which one or more words are input. According to an embodiment, based on the execution of the sentence selector 340, the electronic device 101 may identify one or more sentences included in the media content by grouping one or more words based on the spatial information and the time information. According to an embodiment, the reference Figure 7 The operation of the electronic device 101 to recognize one or more sentences based on the spatial information and the temporal information based on the sentence selector 340 is described.

[0075] In an embodiment, the result of identifying a sentence based on the execution of the sentence selector 340 may include information indicating the position of the sentence in the media content. This information may be stored with the media content, or may be stored in metadata corresponding to the media content. According to an embodiment, the electronic device 101 may display a visual object indicating at least one sentence in the media content based on the information obtained by identifying at least one sentence using the sentence selector 340. For example, based on a rectangular (rect) data structure, the information may represent a rectangle that overlaps with a portion of the media content where the sentences are arranged. The rectangular data structure may include four integers and / or floating-point numbers (wherein the four integers and / or floating-point numbers represent the x-coordinate and y-coordinate of one corner of the rectangle), as well as the width and height of the rectangle. The embodiment is not limited thereto, and the electronic device 101 may store one or more sentences identified by the sentence selector 340 based on other data structures different from the rectangular data structure. For example, based on a combination of nodes included in a word list, the electronic device 101 may store the result of identifying one or more sentences by the sentence selector 340.

[0076] As described above, according to an embodiment, the electronic device 101 may receive an input for identifying handwriting represented by a plurality of strokes in media content. Displaying the media content by the electronic device 101 may be performed based on the execution of the note application 320. In response to the input, the electronic device 101 may recognize a plurality of words from the plurality of strokes based on the execution of the handwriting recognizer 330. The electronic device 101 may recognize at least one sentence from the plurality of words based on the execution of the sentence selector 340. According to an embodiment, the electronic device 101 may improve the user experience associated with handwriting recognition by recognizing or visualizing at least one sentence recognized by the user. For example, the electronic device 101 may support recognizing the strokes displayed in the display 210 in units of sentences recognized by the sentence selector 340.

[0077] In the following, reference Figure 4 According to an embodiment, an example of an operation in which the electronic device 101 receives media content from a user will be described.

[0078] Figure 4 An example of an operation in which the electronic device 101 obtains media content 460 including at least one stroke based on interaction between the user and the electronic device 101 according to an embodiment is shown. Figure 4 The electronic device 101 may be Figures 1 to 3 An example of an electronic device 101. For example, Figure 3 The electronic device 101 and the display 210 may include Figure 4An electronic device 101 and a display 210. Hereinafter, a screen may refer to a user interface (UI) displayed within at least a portion of a display. For example, for explanation purposes, a screen may include an activity of an Android operating system, but may be analogically applied to any other operating system of an electronic device having a display. Figure 4 The screen of the electronic device 101 may be based on Figure 3 An example of a screen displayed in the display 210 due to execution of the note application 320.

[0079] refer to Figure 4 According to an embodiment, the electronic device 101 is shown based on an application for outputting media content 460 (eg, Figure 3 An example of a screen displayed by executing the note application 320 of the electronic device 101. In the area 410 of the screen, the electronic device 101 may display a screen for switching to the application for outputting media content 460. Figure 4 In area 230 of the screen, electronic device 101 may display text corresponding to the title of media content 460, similar to Figure 2 In area 430 of the screen, the electronic device 101 may display at least one visual object for switching from an application in an active state to another application. For example, in area 430 of the screen, the electronic device 101 may display at least one home button for executing a launcher application, a switch button for executing a multitasking-related function based on a list of one or more applications being executed by the electronic device 101, and a return button for returning to another screen displayed before a screen displayed in the display 210. The portion of the screen on which the electronic device 101 displays area 430 may be referred to as the lower end of the screen.

[0080] refer to Figure 4 , the electronic device 101 may display at least a portion of the media content 460 in the area 420 of the screen, similar to Figure 2 region 420. Based on a gesture performed by an external object (e.g., a fingertip) touching a portion of the display 210 on which the region 420 is displayed, the electronic device 101 may change at least a portion of the media content 460 displayed in the region 420. For example, in response to a drag gesture and / or a slide gesture performed within the region 420, the electronic device 101 may scroll the media content 460 displayed in the region 420 based on the direction of the drag gesture and / or the slide gesture. For example, in response to a pinch-to-zoom gesture performed within the region 420, the electronic device 101 may adjust the magnification of the media content 460 displayed in the region 420 based on the direction of the pinch-to-zoom gesture.

[0081] According to an embodiment, the electronic device 101 may display visual objects 442, 444, 446, and 448 for generating and / or editing the media content 460 displayed in the area 420. Figure 4 , showing an example in which the electronic device 101 displays visual objects 442, 444, 446, and 448 in another area different from the area 420, but the embodiment is not limited thereto. For example, the electronic device 101 may display at least one of the visual objects 442, 444, 446, and 448 by at least partially overlapping the area 420. Figure 4 Unlike the embodiment in which all visual objects 442 , 444 , 446 , and 448 are displayed, the electronic device 101 may selectively display at least one of the visual objects 442 , 444 , 446 , and 448 .

[0082] Based on visual objects 442, 444, 446, and 448, electronic device 101 may receive different inputs for editing media content 460. For example, visual object 442 may correspond to a function for editing text entered in media content 460 or adding text in media content 460. In response to input indicating that visual object 442 is selected, electronic device 101 may display a software keyboard within at least a portion of display 210. Based on a touch input performed within the software keyboard, electronic device 101 may add the text indicated by the touch input to a portion of media content 460 (e.g., a portion where a cursor is displayed). For example, visual object 446 may correspond to a function for adding multimedia content (e.g., an image and / or video) such as image 470 to media content 460. In response to input indicating that visual object 446 is selected, electronic device 101 may execute a function for browsing a memory stored in electronic device 101 (e.g., Figure 2 another application (e.g., a gallery application) for multimedia content in the memory 130 of the electronic device 101, or for capturing multimedia content from a camera (e.g., Figure 1 Another application (e.g., a camera application) for obtaining video and / or images through camera module 180 of the electronic device 101. For example, visual object 448 may correspond to a function for adding audio (e.g., the audio represented by icon 250) within media content 460. In response to input indicating selection of visual object 448, electronic device 101 may initiate obtaining an audio signal by using a microphone.

[0083] refer to Figure 4, the electronic device 101 may display a visual object 444 corresponding to a function for receiving an image and / or handwriting based on an external object (e.g., a fingertip and / or a pen 450) in contact with the display 210. The pen 450 may include an external electronic device, such as a digitizer and / or a stylus, for controlling the electronic device 101 based on the contact between the display 210 and the pen 450. The electronic device 101 may identify at least one of the position at which the pen 450 contacts the display 210, the pressure at which the pen 450 presses the display 210, and / or the angle between the display 210 and the pen 450. In an embodiment, a communication link may be established between the electronic device 101 and the pen 450 to identify at least one of the position, pressure, and / or angle. However, the embodiments are not limited thereto.

[0084] According to an embodiment, the electronic device 101 may receive an input for generating or editing the media content 460 based on the visual objects 442, 444, 446, and 448. Figure 4 , showing an example of media content 460 obtained by the electronic device 101 from the user based on input. The electronic device 101 can recognize a stroke drawn by the user based on a single contact between the pen 450 and the display 210. Based on the user drawing multiple strokes in sequence, the electronic device 101 can display handwriting based on the multiple strokes drawn by the user in the area 420. Figure 4 , the media content 460 may include an example of handwriting based on sequentially drawing a plurality of strokes in the area 420. According to an embodiment, the electronic device 101 may store at least one of a time, a timestamp, and / or an index at which each of the plurality of strokes is input, and a location at which each of the plurality of strokes is drawn in the media content 460.

[0085] As described above, according to an embodiment, the electronic device 101 may recognize a plurality of strokes representing handwriting from a user based on an external object (such as a pen 450 and / or a user's finger). The electronic device 101 may store the result of recognizing the plurality of strokes in the media content 460. In a state where the media content 460 is displayed, the electronic device 101 may extract characters, words, and / or sentences distinguished by the plurality of strokes.

[0086] In the following, reference Figure 5 According to an embodiment, an operation of the electronic device 101 recognizing a plurality of strokes included in the media content 460 will be described.

[0087] Figure 5 Examples of words, characters and / or sentences recognized by an electronic device from a plurality of strokes according to an embodiment are shown. Figure 5 An electronic device can be Figures 1 to 4 An example of an electronic device 101. For example, Figure 3 The electronic device 101 may include Figure 5 electronic equipment. Figure 5 The media content 460 may correspond to Figure 4 460 of media content.

[0088] refer to Figure 5 , exemplarily showing media content 460 obtained by the electronic device based on the strokes drawn by the user. Figure 4 After visual object 444 in the image, the user can use a pen (e.g., Figure 4 The pen 450 in the figure draws strokes indicating handwriting. While the user draws the strokes, the electronic device can display the display (e.g., Figure 3 The electronic device may obtain a list of one or more points for reconstructing a stroke by connecting at least one pixel in the display 210 in the pen. For example, the electronic device may obtain the list of points by connecting nodes corresponding to each point included in the stroke. Each node may include at least one of the position of the corresponding point and / or the time when the point was drawn by the pen (e.g., a millisecond-based timestamp). The electronic device may store the list of points in the media content 460.

[0089] According to an embodiment, the electronic device may recognize at least one stroke from the media content 460. Figure 5 , exemplarily showing strokes 512 and 514 recognized by the electronic device from the media content 460. The strokes 512 and 514 may be recognized based on the pen (e.g., Figure 4 Each of the strokes 512 and 514 is recognized by a single contact between the pen 450 and the electronic device. According to an embodiment, the electronic device may display the stroke 512 based on a list of points corresponding to the stroke 512. For example, the electronic device may display the stroke 512 on a display (e.g., Figure 3 The points included in the list are displayed in the display 210 of the electronic device, and the stroke 512 is displayed by displaying other points and / or lines based on the interpolation of the points. Displaying the stroke 514 in the media content 460 by the electronic device can be performed similarly to the operation of displaying the stroke 512 described above.

[0090] According to an embodiment, the electronic device may recognize the character 510 including the strokes 512 and 514 based on the position and / or order of drawing the strokes 512 and 514. Recognition of the character 510 by the electronic device may include recognizing the letter represented by the combination of the strokes 512 and 514 (e.g., Figure 4 According to an embodiment, the electronic device may recognize a plurality of characters including the character 510 from a plurality of strokes included in the media content 460. For example, a plurality of strokes may be grouped into different characters.

[0091] According to an embodiment, the electronic device may recognize a word from a plurality of characters including the character 510 within the media content 460. Figure 5 , exemplarily showing words 242-1, 242-2, 242-3, and 242-4 recognized by the electronic device from media content 460. For example, the electronic device may recognize the spacing for distinguishing words in media content 460 based on the spacing between characters within media content 460. Based on the spacing for distinguishing words in media content 460, the electronic device may recognize words from a plurality of characters. For example, similar to a plurality of strokes being grouped into different characters, characters recognized from a plurality of strokes in media content 460 may be grouped into different words.

[0092] According to an embodiment, the electronic device may identify the positions of a plurality of words identified from a plurality of strokes in the media content 460 and / or the moment when the plurality of words are input. For example, the electronic device may obtain the coordinates of each of the plurality of words in the media content 460. The coordinates may be representative values ​​(e.g., minimum, maximum, average, and / or median) of the coordinates of the strokes included in the corresponding word in the media content 460. For example, the electronic device may obtain a timestamp indicating the moment when each of the plurality of words is input. The moment when a word is input may be a representative value (e.g., minimum, maximum, average, and / or median) of the timestamp assigned to each point included in the strokes included in the word. The moment when a word is input may include an index indicating the position of the word within the sequence of the plurality of words.

[0093] According to an embodiment, the electronic device may recognize at least one sentence based on a word recognized by grouping a plurality of strokes. Figure 5 , the electronic device can recognize the sentence 242 distinguished by the words 242-1, 242-2, 242-3 and 242-4. The electronic device can recognize a plurality of sentences in the media content 460, including the sentence 242, based on the spacing between the words included in the media content 460 and the time when the words are input.

[0094] In the following, reference Figure 6 According to an embodiment, an example of an operation of an electronic device recognizing a plurality of sentences based on a spacing between words and a time when a word is input will be described.

[0095] Figure 6 An example of an operation of an electronic device recognizing a distance between words distinguished by a plurality of strokes according to an embodiment is shown. Figure 6 An electronic device can be Figures 1 to 5 Examples of electronic devices. For example, Figure 3 The electronic device 101 may include Figure 6 electronic equipment. Figure 6The media content 460 may correspond to Figures 4 to 5 460 of media content.

[0096] According to an embodiment, the electronic device may recognize a spatial structure (or spatial information) and / or a temporal structure (or temporal information) of a plurality of words distinguished by a plurality of strokes included in the media content 460. The spatial structure may indicate the spacing between the plurality of words, and / or the indentation of at least one of the plurality of words. The temporal structure may indicate the moment and / or order of inputting the plurality of words. According to an embodiment, the recognition of the spatial structure and / or the temporal structure by the electronic device may be performed based on the execution of a function (e.g., smart selection) for identifying or selecting characters, words, and / or sentences. For example, based on the execution of the function, the electronic device may Figure 3 The handwriting recognizer 330 is executed to obtain spatial information and / or temporal information of the plurality of words.

[0097] According to an embodiment, the electronic device may group multiple words in units of sentences based on the spatial information and time information of the multiple words. For example, the electronic device may obtain a candidate sentence including at least one of the multiple words in the media content 460 and a probability assigned to the candidate sentence. The probability may indicate the probability that the candidate sentence is a single sentence. The candidate sentence may be referred to as a set of one or more words. According to an embodiment, the electronic device may group each of the multiple words based on the probability assigned to the candidate sentence. For example, the electronic device may determine the candidate sentence with a probability exceeding a threshold probability as a sentence identified by the media content 460. According to an embodiment, the electronic device may use spatial information and time information to identify the candidate sentence. In one or more embodiments, the electronic device 101 may include software for determining the probability that the candidate sentence is a single sentence. The software may include computer executable instructions processed by the processor 120. The software for determining the probability that the candidate sentence is a single sentence may include one or more algorithms based on: heuristic, rule-based logic and / or training algorithms to form a machine learning model. When the algorithm is trained to classify a candidate sentence as a single sentence or not as a single sentence, or when the algorithm is trained to output a probability value, the training includes training the algorithm using training data, wherein the training data includes labels / probability values ​​for a plurality of candidate sentences, each candidate sentence having a plurality of words based on spatial information and temporal information of the plurality of words. In one or more embodiments, as understood by a person of ordinary skill in the art, the software may include and / or call any known software application for determining the probability that a candidate sentence is a single sentence.

[0098] In the following, reference Figure 6According to an embodiment, an example of an operation in which an electronic device obtains time information of a plurality of words will be described. According to an embodiment, the electronic device may obtain time information including the moment of inputting each of the plurality of words and / or an index assigned to each of the plurality of words based on these moments. According to an embodiment, the electronic device may obtain the moment and / or timestamp of inputting all strokes included in the media content 460 from the media content 460. The electronic device may obtain the moment and / or timestamp of inputting each word as a group of strokes based on the moment and / or timestamp of inputting the strokes. Within the time information, the timestamp assigned to the word may be a representative value (e.g., a minimum value, a maximum value, an average value, and / or a median value) of the timestamps of the strokes included in the input word.

[0099] refer to Figure 6 According to an embodiment, the electronic device may obtain the moment and / or order of inputting each word from the time information included in the media content 460. When a word is included in another sentence, the difference between the moments of sequentially inputting the interconnected words may be relatively increased. For example, the difference between the moment of the word included in sentence 241 (e.g., "Hello!") and the first word in sentence 242 recorded immediately following sentence 241 (e.g., word 242-1 indicating "nice") may exceed the difference between the moments of recording words 242-1 and 242-2 in sentence 242. According to an embodiment, the electronic device may classify the words into different sentences based on the time information including the moment and / or order of inputting the words. In the following, it is assumed that the time information is recorded as words 242-1, 242-2, 242-3, and 242-4 included in sentence 242, words 244-1 and 244-2 included in sentence 244, and words 243-1, 243-2, 243-3, and 243-4 (e.g., in the following) Figure 7 The order of sentences described in 243).

[0100] In the following, reference Figure 6 According to an embodiment, an example of an operation in which an electronic device obtains spatial information of a plurality of words will be described. According to an embodiment, the electronic device may obtain spatial information (including the spacing between a plurality of words), and the position of the words in the media content 460 (e.g., indentation, margin, and / or padding). The spatial information may include the spacing between the plurality of words measured in a first direction in which the plurality of words are sequentially input and in a direction different from the first direction (e.g., a second direction perpendicular to the first direction). In the following, the first direction may be a direction connecting at least two characters included in any one of the plurality of words. For example, the first direction may be parallel to a direction of at least one row formed by the plurality of words.

[0101] refer to Figure 6, according to an embodiment, spacings 611, 612, 613, and 614 recognized by the electronic device from words 242-1, 242-2, and 242-3 in a single line formed along a first direction are shown. The electronic device can obtain the spacings 611, 612, 613, and 614 based on the areas occupied by the words 242-1, 242-2, and 242-3 in the media content 460. These areas can be based on the shape of the graphic (e.g. Figure 6 A dotted square (indicated by a dashed line in FIG. 1 ) surrounding each word 242-1, 242-2, and 242-3 is formed in the media content 460. For example, the electronic device can obtain these areas from the words 242-1, 242-2, and 242-3 based on the rectangular data structure.

[0102] refer to Figure 6 , spacing 611 may be the margin of word 242-1, which is drawn first among words 242-1, 242-2, and 242-3. Spacing 612 may be the distance between words 242-1 and 242-2. Spacing 613 may be the distance between words 242-2 and 242-3. Spacing 614 may be the margin formed in media content 460 by word 242-3, which is drawn last among words 242-1, 242-2, and 242-3. Spacing 611, 612, 613, and 614 may exceed (i.e., be greater than) the spacing between characters in each word 242-1, 242-2, and 242-3.

[0103] Similar to the operation of identifying spacings 611, 612, 613, and 614 based on words 242-1, 242-2, and 242-3, the electronic device can obtain the spacings between other words in the media content 460 based on the first direction. For example, spacings 621 and 622 can be the margins of word 242-4. Spacing 631 can be the margin of word 243-1. Spacing 632 can be the spacing between words 243-1 and 243-2. ​​Spacing 641 can be the margin of word 244-1. Spacing 642 can be the spacing between words 244-1 and 244-2. Spacing 643 can be the margin formed by word 244-2 in the media content 460.

[0104] According to an embodiment, the electronic device may identify a spacing between a plurality of rows based on a spacing between words included in each of the plurality of rows divided in a second direction perpendicular to the first direction. Figure 6Spacings 651, 652, and 653 may exemplarily represent spacings between multiple rows. For example, spacing 651 may be the distance between words 242-1 and 242-4 in the second direction. Spacing 651 may be the distance between a first row formed by words 242-1, 242-2, and 242-3 and a second row formed by word 242-4. Spacing 652 may be the distance between a second row formed by word 242-4 and a third row formed by words 243-1, 243-2, 243-3, and 243-4.

[0105] refer to Figure 6 , the spacing 653 can be the distance between the second row formed by the word 242-4 and the fourth row formed by the words 244-1 and 244-2. Based on the use time information, the electronic device can obtain the spacing 653 between the second row including the word 242-4 and the fourth row including the word 244-1 by recognizing that the word 244-1 is recorded immediately after the word 242-4 in the second row is recorded based on the use time information. The recording of the word 244-1 immediately after the word 242-4 in the second row can be identified based on the direct connection of the word 242-4 and the word 244-1 within the sequence indicated by the time information. According to an embodiment, the electronic device can obtain the spacing based on the sequence of words indicated by the time information. For example, the electronic device can obtain the spacing 653 between the words 242-4 and 244-1 directly connected within the sequence.

[0106] According to an embodiment, the electronic device may obtain the spacing between a plurality of words and / or a plurality of rows independently of the unit (e.g., page) used to divide the media content 460 to display or print the media content 460. For example, in a state where the word 242-4 and the word 244-1 are displayed on different pages, the electronic device may obtain the spacing 653 between the word 242-4 and the word 244-1 independently of the distance between the pages and / or the arrangement. In an embodiment, the electronic device may exclude the width and / or height of the multimedia content (e.g., the image 470 and / or the audio corresponding to the icon 250) included in the media content 460 from the spacing between the words (e.g., when determining the spacing between the words).

[0107] According to an embodiment, the spatial information obtained by the electronic device from the words in the media content 460 may include data about the spacing between words and / or lines formed along the first direction and / or the second direction, such as spacings 611, 612, 613, 614, 621, 622, 631, 632, 641, 642, 643, 651, 652, and 653. The spatial information may include one or more weights based on the spacing. For example, the electronic device may assign a weight to each spacing based on whether each spacing exceeds a specified threshold. Each weight included in the spatial information may indicate the probability that the spacing corresponding to the weight is the spacing formed between sentences. According to an embodiment, the electronic device may identify at least one candidate sentence based on the data and / or weight included in the spatial information.

[0108] According to an embodiment, the electronic device may use time information and spatial information to identify at least one sentence. For example, when the spacing 622 formed by the word 242-4 exceeds the specified spacing or when the difference between the moment when the word 242-4 is input and the moment when the word 244-1 is recorded immediately after the word 242-4 is input exceeds the specified time difference, the electronic device may determine that the words 242-4 and 244-1 are included in different sentences 242 and 244. For example, the electronic device may identify the word 242-4 as the last word of the sentence 242. For example, when the spacing 651 between the first and second rows is less than the spacings 652 and 653 formed by the second row and other rows, or when the spacing 651 is less than or equal to the specified spacing, the electronic device may determine that the first and second rows adjacent to the spacing 651 are a single sentence 242.

[0109] As described above, according to an embodiment, the electronic device can identify whether a word is included in different sentences based on at least one of the spacing between words, the time when the word is input, or the difference between the times. The spacing between words can include the spacing between words directly connected within the order of inputting the word.

[0110] According to an embodiment, the electronic device may recognize at least one sentence from the media content 460 using a model trained to recognize a plurality of strokes in units of sentences. Figure 7 According to an embodiment, an operation of an electronic device to recognize at least one sentence using a model will be described.

[0111] Figure 7 An example of an operation of the electronic device recognizing a sentence 240 from a plurality of strokes according to an embodiment is shown. Figure 7 An electronic device can be Figures 1 to 6 Examples of electronic devices. For example, Figure 3 The electronic device 101 may include Figure 7 For example, Figure 3The sentence selector 340 may include Figure 7 A sentence selector 340 is provided. Figure 7 The media content 460 may correspond to Figures 4 to 6 460 of media content.

[0112] refer to Figure 7 According to an embodiment, the electronic device may recognize the sentence 240 from the strokes included in the media content 460 based on the sentence selector 340. In an embodiment, the sentence selector 340 may include a model that simulates the computing power of a biological nervous system using a plurality of artificial neurons (or nodes). The model may operate based on software such as the sentence selector 340 and / or hardware such as a neural processing unit (NPU), the software being based on data stored in a memory (e.g., Figure 3 Instructions in memory 130). Figure 7 , the semantic analyzer 710 and the spatiotemporal analyzer 720 are shown as models included in the sentence selector 340. The embodiment is not limited thereto, and for example, the electronic device may be based on the execution of the spatiotemporal analyzer 720, based on the above reference Figure 6 The described deterministic algorithm for obtaining temporal and spatial information operates.

[0113] According to an embodiment, the electronic device may obtain a sentence 240 from a plurality of strokes in the media content 460 based on a semantic analyzer 710, the semantic analyzer including a model trained to extract text from a single sentence of a plurality of strokes. The model may be a neural network based on a long short-term memory (LSTM) and / or a recurrent neural network (RNN). For example, the semantic analyzer 710 may include a model trained based on input data, the input data including one or more strokes divided in units of sentences, text labels identified from one or more strokes, and / or real information. The electronic device may use the model of the semantic analyzer 710 to output information representing one or more sentences from a plurality of strokes.

[0114] According to an embodiment, the electronic device may obtain time information and / or space information of a word divided by a plurality of strokes based on the time-space analyzer 720. Using the time-space analyzer 720, the electronic device may obtain space information indicating the spacing between words and time information indicating the moment and / or order of inputting words from words in the media content 460. In an embodiment, the time-space analyzer 720 may include a model trained to obtain time information and / or space information.

[0115] According to an embodiment, the electronic device may recognize the sentence 240 in the media content 460 based on the information obtained from the semantic analyzer 710 and the time information and space information obtained from the spatiotemporal analyzer 720. For example, the electronic device may recognize the sentence 240 based on a model (or attention layer) that receives the information obtained from the semantic analyzer 710 and the time information and space information obtained from the spatiotemporal analyzer 720. Figure 7 , showing sentences 241, 242, 243, 244, 245, 246, 247, 248, and 249 obtained by the electronic device using the sentence selector 340 including the semantic analyzer 710 and the spatiotemporal analyzer 720. According to an embodiment, the electronic device may respond to an input for selecting a specific sentence within the handwriting represented by a plurality of strokes using the recognized sentences 241, 242, 243, 244, 245, 246, 247, 248, and 249.

[0116] As described above, according to an embodiment, the electronic device may recognize one or more sentences through a group of words based on time information and / or spatial information about words distinguished by strokes included in the media content 460. The electronic device may use a model trained by handwriting divided into sentence units (e.g., a model of the semantic analyzer 710) and time information and spatial information to improve the accuracy of recognizing sentences. According to an embodiment, in response to an input indicating that a stroke is selected as a sentence unit in the media content 460, the electronic device may obtain at least one sentence including the stroke selected by the input. The electronic device may highlight at least one sentence, or output text indicating at least one sentence.

[0117] In the following, reference Figures 8a to 8c , Figure 9a to Figure 9b and / or Fig.10 According to an embodiment, an example of an operation in response to an input instructing an electronic device to select a stroke as a sentence unit will be described.

[0118] Figures 8a to 8c An example of an operation of an electronic device recognizing a word, a character, and / or a sentence based on a user's gesture according to an embodiment is shown. Figures 8a to 8c The electronic device 101 may be Figures 1 to 7 Examples of electronic devices. For example, Figure 3 The electronic device 101 and the display 210 may include Figures 8a to 8c The electronic device 101 and the display 210. For example, Figures 8a to 8c The states 810, 820 and 830 may be respectively the electronic device 101 based on Figure 3 The different states of the display screen are displayed according to the execution of the note application 320. For example, Figure 4 The regions 410, 420 and 430 may respectively include Figures 8a to 8cregions 410 , 420 and 430 .

[0119] refer to Figure 8a , displaying at least a portion of the media content in area 420 (e.g., Figures 4 to 7 In state 810 of selecting media content 460 (e.g., "meet") displayed in point 805 within area 420, electronic device 101 may receive input indicating selection of a word (e.g., "meet") displayed in point 805 within area 420. The input may include a double-click gesture of clicking point 805 twice within a specified time period. Although the operation of electronic device 101 for input based on point 805 is described exemplarily, the embodiments are not limited thereto. For example, based on a double-click gesture performed on another word in the media content displayed through area 420, electronic device 101 may receive another input indicating selection of another word.

[0120] According to an embodiment, the electronic device 101 may recognize at least one word 242-3 including the strokes displayed in the point 805 in response to an input indicating selection of the word displayed in the point 805. For example, the electronic device 101 may recognize at least one word 242-3 including the strokes displayed in the point 805 based on Figure 3 The handwriting recognizer 330 is executed to recognize the word 242-3 including the stroke represented by the point 805 specified by the input.

[0121] refer to Figure 8a Based on recognizing the word 242-3 according to the input, the electronic device 101 may display a visual object 812 for highlighting the word 242-3 in the area 420. For example, the electronic device 101 may display the visual object 812 having a rectangular shape surrounding the word 242-3. The shape of the visual object 812 is not limited to Figure 8a According to an embodiment, the electronic device 101 may display the word 242 - 3 based on a color (eg, blue, red) different from other words included in the media content or may invert the color of the word 242 - 3 .

[0122] According to an embodiment, in state 810 of recognizing or selecting word 242-3 based on visual object 812, electronic device 101 may receive input indicating adjustment of characters and / or words selected in media content based on visual object 812. For example, based on input indicating expansion or reduction of visual object 812, electronic device 101 may recognize at least one character included in visual object 812 that is expanded or reduced by the input.

[0123] According to an embodiment, the electronic device 101 may identify a single line distinguished by strokes in the media content based on another input different from the input indicating selection of a word in the media content. Figure 8bAccording to an embodiment, a state 820 of recognizing a single line including a word displayed in point 805 in response to electronic device 101 receiving other input based on point 805 in area 420 is exemplarily shown. For example, other input may include a triple-tap gesture of clicking point 805 three times within a specified time period.

[0124] According to an embodiment, in response to an input indicating selection of a single row, such as a triple-tap gesture performed on point 805, the electronic device 101 may identify a single row including strokes displayed within point 805 selected by the input within the media content. The electronic device 101 may display a visual object 822 indicating that the identified row is selected. Figure 8b Based on the visual object 822 having a rectangular shape, the electronic device 101 may highlight the line including the stroke represented in the point 805 (eg, "nice to meet") instead of other lines. The shape of the visual object 822 is not limited to Figure 8b The electronic device 101 may display the strokes in the row identified based on the input in a different color from other strokes in the media content.

[0125] According to an embodiment, the electronic device 101 may receive an input indicating selection of a single sentence, which is similar to the above reference Figure 8a to Figure 8b The input described is different. Figure 8c , the electronic device 101 may receive an input indicating a selection of a single sentence including the strokes included in the point 805. The input may include a four-tap gesture of tapping the point 805 four times within a specified time period. In response to the input, the electronic device 101 may Figure 3 The handwriting recognizer 330 and / or the sentence selector 340 are executed to recognize the sentence 242 including the stroke represented in point 805.

[0126] refer to Figure 8c , in response to an input indicating selection of a sentence based on point 805, a state 830 in which the electronic device 101 recognizes the sentence 242 including the stroke represented within point 805 is exemplarily shown. The electronic device 101 may display a visual object 832 for highlighting the sentence 242. The visual object 832 may indicate recognition or selection of the sentence 242 based on the shape of a rectangle, similar to Figure 8a to Figure 8b 812 and 822. However, the shape of the visual object 832 is not limited thereto. According to an embodiment, the electronic device 101 may display the strokes included in the sentence 242 (e.g., the strokes representing “nice to meet you”) in a color different from other strokes displayed in the area 420 to indicate that the sentence 242 is recognized or selected.

[0127] According to the embodiment, the electronic device 101 can be based on Figures 8a to 8c The electronic device 101 may further display visual objects 812, 822, and 832 to receive input indicating movement and / or rotation of the words, lines, and sentences designated by the visual objects 812, 822, and 832. For example, the electronic device 101 may move the strokes included in the visual object 832 within the media content based on a gesture of dragging the visual object 832. For example, the electronic device 101 may rotate the strokes included in the visual object 832 in the media content based on a gesture of rotating the visual object 832. According to an embodiment, the electronic device 101 may additionally display a visual object in order to perform a function based on text corresponding to the words, lines, and sentences designated by the visual objects 812, 822, and 832.

[0128] According to an embodiment, the electronic device 101 may perform a function of recognizing or selecting strokes in units of sentences and in units of words and lines. Figure 9a to Figure 9b According to an embodiment, an example of an operation in which an electronic device recognizes a sentence and provides a UI in order to execute a function related to a text corresponding to the recognized sentence is described.

[0129] According to an embodiment, Figure 9a to Figure 9b An example of an operation of the electronic device 101 recognizing a sentence from media content including a plurality of strokes is shown. Figure 9a to Figure 9b The electronic device 101 may be Figures 1 to 7 Electronic equipment and / or Figures 8a to 8c An example of an electronic device 101. For example, Figure 3 The electronic device 101 and the display 210 may include Figure 9a to Figure 9b The electronic device 101 and the display 210. For example, Figure 9a to Figure 9b The states 910 and 920 can be respectively the electronic device 101 based on Figure 3 The note application 320 is executed to display different states of the screen. For example, Figure 4 Regions 410, 420, and 430 may include Figure 9a to Figure 9b regions 410 , 420 and 430 .

[0130] refer to Figure 9a, in a state 910 where at least a portion of the media content is displayed in the area 420 of the display 210, the electronic device 101 may display handles 911 and 912 for identifying strokes in the media content in units of characters and / or words. For example, the electronic device 101 may display at least one character distinguished by strokes based on information indicating strokes sequentially drawn by a user in the media content. The electronic device 101 may perform the display of the handles 911 and 912 in response to an input indicating that the handles 911 and 912 are to be displayed (such as a long touch gesture performed in the area 420). Each of the handles 911 and 912 may respectively represent the beginning and end of at least one character and / or word selected by the user in the characters and / or words distinguished by strokes. Based on an input indicating dragging any one of the handles 911 and 912, the electronic device 101 may change at least one character and / or word selected by the user based on the position of the handles in the input dragging the handles 911 and 912.

[0131] refer to Figure 9a In state 910 where a stroke representing a specific word (e.g., a stroke representing "brown") is recognized based on handles 911 and 912, electronic device 101 may display menu 914 in order to execute a function related to the text displayed by the stroke between handles 911 and 912. According to an embodiment, electronic device 101 may display menu 914 in area 913 of display 210 in state 910 where handles 911 and 912 are displayed. Electronic device 101 may display at least one character (e.g., a character) recognized by handles 911 and 912 by using menu 914. Figure 9a A list of functions related to the state "brown" in state 910).

[0132] refer to Figure 9a , the functions provided by menu 914 may include a function for copying at least one character recognized based on handles 911 and 912 (e.g., a function corresponding to an option including specified text (e.g., "copy"). Copying at least one character may include at least temporarily storing the text represented by the stroke between handles 911 and 912 in a memory (e.g., Figure 3The functions provided through menu 914 may include a function for cutting at least one character identified based on handles 911 and 912 (e.g., a function corresponding to an option including specified text (e.g., "Cut")). Cutting at least one character may include at least temporarily stopping the display of the strokes between handles 911 and 912. The functions provided through menu 914 may include a function for inserting text stored in a specified area of ​​memory (e.g., a clipboard) into a portion of the media content identified by handles 911 and 912 (e.g., a function corresponding to an option including specified text (e.g., "Paste")). The functions provided through menu 914 may include a function for deleting the strokes between handles 911 and 912 from the media content (e.g., a function corresponding to an option including specified text (e.g., "Delete")).

[0133] refer to Figure 9a In the state 910 where the strokes between the handles 911 and 912 do not match a single sentence, the electronic device 101 may display a visual object 915 in a menu 914 for identifying at least one sentence including the strokes between the handles 911 and 912. In the menu 914, other visual objects except the visual object 915 may correspond to functions related to at least one character selected by the handles 911 and 912. The visual object 915 may include a designated text for guiding the recognition of the sentence, such as "Select a sentence". In response to the Figure 9a In the state 910 of selecting the input of the visual object 915, the electronic device 101 can Figure 3 The handwriting recognizer 330 and / or the sentence selector 340 of the embodiment can be used to identify the sentence including the stroke between the handles 911 and 912 in the media content. Figures 5 to 7 As described above, the electronic device 101 may perform sentence recognition based on the distance between words distinguished by strokes included in the media content and / or the time when a word is input.

[0134] refer to Figure 9b , according to an embodiment, an example of state 920 is shown, in which the electronic device 101 responds to the indication selection Figure 9a The electronic device 101 can identify at least one sentence 243 in the media content based on the input of the visual object 915. For example, in response to the input, the electronic device 101 can identify the at least one sentence 243 including Figure 9a The sentence 243 of the strokes between the sentence handles 911 and 912 (e.g., “the quick brown fox jumps over the lazy dog”). Figure 9bIn the embodiment, the electronic device 101 may display handles 911 and 912 in the media content at the point where the sentence 243 starts and the point where the sentence 243 ends, respectively. The electronic device 101 may display the sentence 243 in a color different from other sentences in the media content.

[0135] refer to Figure 9b According to an embodiment, the electronic device 101 may display a menu 922 in association with at least one sentence included in the sentence 243 in a state where the sentence 243 is recognized. For example, the menu 922 may be overlapped on a portion 921 adjacent to the sentence 243 within the area 420 of the display 210. According to an embodiment, in the state 920 where a single sentence such as the sentence 243 is recognized, the electronic device 101 may at least temporarily stop displaying a visual object (e.g., Figure 9a 915). Figure 9b As shown, when a single sentence is recognized through handles 911 and 912, the display of the visual object may be paused. Figure 9a Menu 914 and Figure 9b The menu 922 may maintain display of visual objects other than the visual objects mapped to the functions for guiding the recognition of a single sentence.

[0136] The operation of the electronic device 101 providing a list of functions related to the strokes recognized by the handles 911 and 912 is not limited to Figure 9a to Figure 9b Menus 914 and 922. Reference Figure 9b According to an embodiment, in the state 920 where the sentence 243 is recognized, the electronic device 101 may display a pop-up window 930 including a text 932 corresponding to the sentence 243. The electronic device 101 may Figure 9b The pop-up window 930 is displayed by overlapping the display 210 on the screen. In the pop-up window 930, the electronic device 101 can display the text 932 represented by the strokes included in the sentence 243. The electronic device 101 can display an icon 938 for stopping the display of the pop-up window 930 in the pop-up window 930. The electronic device 101 can display a button 934 in the pop-up window 930 to at least temporarily store the text 932 in the memory. Figure 9b , button 934 may include designated text, such as “Copy”. In response to input indicating that button 934 has been selected, electronic device 101 may stop displaying pop-up window 930. In response to input indicating that button 934 has been selected, electronic device 101 may at least temporarily store text 932 in a designated area of ​​memory (e.g., a clipboard).

[0137] refer to Figure 9b, electronic device 101 may display button 936 in pop-up window 930 to convert strokes in the media content identified by handles 911 and 912 into text 932. Button 936 may include specified text, such as "Convert". In response to input indicating that button 936 is selected, electronic device 101 may remove the strokes identified by handles 911 and 912 from the media content and insert text 932 into at least a portion of the media content corresponding to handles 911 and 912.

[0138] As described above, according to an embodiment, in the state 910 in which at least one character and / or at least one word distinguished by the strokes between the handles 911 and 912 is recognized, the electronic device 101 may expand the range of the strokes selected by the user based on the sentence unit. For example, the electronic device 101 may display an option to expand the range based on the sentence unit, such as the visual object 915. The electronic device 101 may recommend to the user based on the input for selecting a word in the middle of the sentence that the sentence includes the recognized word. Based on the recognition of a single sentence, the electronic device 101 may select a sentence including the recognized word based on the input for selecting a word in the middle of the sentence. Based on the recognition of the single sentence, the electronic device 101 may select a sentence including the recognized word based on the input for selecting a word in the middle of the sentence by the electronic device 101 and / or the processor (e.g., Figure 3 The processor 120 in the processor 120 outputs the text corresponding to the recognized sentence in a format readable by the processor 120 (e.g., Unicode).

[0139] According to an embodiment, the electronic device 101 may additionally identify multimedia content related to the sentence within the media content in response to an input indicating that the sentence is selected. Fig.10 An example of an operation in which the electronic device 101 recognizes multimedia content based on an input is described.

[0140] Fig.10 An example of an operation of an electronic device according to an embodiment to recognize a sentence in media content and an image related to the sentence from the media content including a plurality of strokes is shown. Fig.10 The electronic device 101 may be Figures 1 to 7 , Figures 8a to 8c and Figure 9a to Figure 9b An example of an electronic device 101. For example, Figure 3 The electronic device 101 and the display 210 may include Fig.10 The electronic device 101 and the display 210. For example, Fig.10 The state 1010 may include the electronic device 101 based on Figure 3 The note application 320 is executed to display the status of the screen. For example, Figure 4 Regions 410, 420, and 430 may include Fig.10 regions 410 , 420 and 430 .

[0141] refer to Fig.10In state 1010, according to an embodiment, the electronic device 101 may display an image 470 included in the media content and the strokes included in the media content. The electronic device 101 may display an icon 250 for reproducing the audio included in the media content and the strokes included in the media content. The image 470 and the audio may be referred to as multimedia content included in the media content. According to an embodiment, the electronic device 101 may search for multimedia content related to the sentence selected by the input in response to an input indicating that a sentence distinguished by strokes is selected. Based on the identification of multimedia content related to the sentence, the electronic device 101 may display a visual object indicating that both the sentence and the multimedia content have been identified.

[0142] refer to Fig.10 , in state 1010, the electronic device 101 may receive an input indicating that the sentence 249 is selected. Based on the input, the electronic device 101 may search for multimedia content related to the sentence 249 within the media content. For example, the electronic device 101 may identify whether the words included in the sentence 249 are related to the audio corresponding to the image 470 and / or the icon 250. When an object is recognized from the image 470, the electronic device 101 may determine whether the image 470 is related to the sentence 249 based on whether the word representing the object is included in the sentence 249. When speech-to-text (STT)-based text is recognized from the audio, the electronic device 101 may determine whether the audio is related to the sentence 249 based on whether at least one word in the sentence 249 is included in the text.

[0143] refer to Fig.10 , based on recognizing that the word included in sentence 249 (e.g., "mountain") represents an object included in image 470 (e.g., mountain), electronic device 101 can recognize both sentence 249 and image 470. For example, electronic device 101 can display visual object 1020 including both sentence 249 and image 470. Although an example in which visual object 1020 has a rectangular shape surrounding both sentence 249 and image 470 is shown, the embodiment is not limited thereto. In an embodiment, electronic device 101 can display sentence 249 and image 470 in a specified color to indicate the user's selection.

[0144] Although the operation of the electronic device 101 is described based on the image 470, the embodiment is not limited thereto. Fig.10 , when the audio corresponding to icon 250 includes sounds (e.g., the sound of a bird) related to a word included in sentence 246 (e.g., "birds"), electronic device 101 may display a visual object indicating identification of both icon 250 and sentence 246 in response to input indicating selection of sentence 246.

[0145] As described above, according to an embodiment, in a state 1010 of displaying media content including strokes representing handwriting, the electronic device 101 may recognize the sentence based on time information and spatial information about the words distinguished by the strokes in response to an input indicating selection of a sentence distinguished by the strokes (e.g., sentences 246 and 249). The electronic device 101 may recognize multimedia content related to the recognized sentence from multimedia content included in the media content. In response to recognizing the multimedia content, the electronic device 101 may display a visual object 1020 indicating that both the sentence and the multimedia content are recognized.

[0146] In the following, reference will be made to Figures 11 to 13 The operation of the electronic device 101 according to the embodiment is described.

[0147] Fig.11 An example of a flowchart for describing an operation of obtaining one or more sentences from a plurality of strokes by an electronic device according to an embodiment is shown. Fig.11 An electronic device can be Figures 1 to 10 An example of an electronic device 101. For example, Fig.11 At least one of the operations may be Figure 3 Executed by the electronic device 101 and / or the processor 120.

[0148] refer to Fig.11 In operation 1110, according to an embodiment, the electronic device may recognize a plurality of strokes within the media content. The media content may include Figures 4 to 7 The plurality of strokes may represent a trajectory drawn sequentially by a user, such as Figure 5 512 and 514. For example, multiple strokes may represent a user's handwriting input. Recognizing multiple strokes by the electronic device may include recognizing one or more words distinguished by the multiple strokes.

[0149] refer to Fig.11 In operation 1120, according to an embodiment, the electronic device may obtain spatial information including the spacing between multiple words distinguished by multiple strokes. The spacing recognized by the electronic device may include the spacing between words, the margin and / or indentation of the words. According to an embodiment, the electronic device may recognize the spacing based on a first direction connecting characters within the word and a second direction perpendicular to the first direction. For example, the spacing in operation 1120 may include spacings 611, 612, 613, 614, 621, 622, 631, 632, 641, 642, 643, 651, 652, and 653.

[0150] refer to Fig.11In operation 1130, according to an embodiment, the electronic device may obtain time information based on the time when a plurality of words distinguished by a plurality of strokes are input. The time information may represent the time based on a timestamp. The time information may include the order in which the plurality of words are input.

[0151] refer to Fig.11 In operation 1140, according to an embodiment, the electronic device may identify a candidate sentence including at least one of the plurality of words based on the spatial information and the temporal information. Figure 3 Sentence selector 340, Figure 7 The electronic device may identify the candidate sentences based on a set including at least one word. The electronic device may obtain the probability that each candidate sentence is a sentence distinguished by multiple words.

[0152] refer to Fig.11 In operation 1150, according to an embodiment, the electronic device may obtain one or more sentences from a plurality of words based on the candidate sentences. For example, the electronic device may obtain one or more sentences based on the probabilities assigned to the candidate sentences. According to an embodiment, in a state where one or more sentences are obtained, the electronic device may perform a function of recognizing at least one sentence including characters and / or words selected by a user based on operation 1150.

[0153] Figure 12a to Figure 12b An example of a flowchart for describing an operation performed by an electronic device based on an input received from a user according to an embodiment is shown. Figure 12a to Figure 12b Electronic devices may include Figures 1 to 11 For example, Figure 12a to Figure 12b At least one operation can be performed by Figure 3 Executed by the electronic device 101 and / or the processor 120. Figure 12a to Figure 12b At least one of the operations can be combined with Fig.11 is related to at least one operation in .

[0154] refer to Fig.12a In operation 1210, according to an embodiment, the electronic device may display media content including a plurality of strokes. For example, the electronic device may display the media content including a plurality of strokes on a display (eg, Figure 3 The electronic device may display at least a portion of the media content in a display 210 of the electronic device. The electronic device may display the plurality of strokes in the display based on information including a time and a position at which each of the plurality of strokes is input and recognized from the media content.

[0155] refer to Fig.12aIn operation 1220, according to an embodiment, the electronic device may receive a first input indicating selection of a group (eg, a character and / or a word) distinguished by at least one of a plurality of strokes. The first input may include a reference Figures 8a to 8c , Figure 9a to Figure 9b and / or Fig.10 The gesture shown.

[0156] refer to Fig.12a In operation 1230, according to an embodiment, the electronic device may display a visual object corresponding to each function related to the group identified by the first input. The visual object may be based on, for example, Figure 9a In operation 1230, the electronic device may display a first visual object so as to recognize a sentence including at least one character selected by the first input, such as Figure 9a As shown in the visual object 915.

[0157] refer to Fig.12a In operation 1240, according to an embodiment, the electronic device may determine whether a second input indicating selection of one or more visual objects in operation 1230 is received. In response to not receiving the second input selecting one or more visual objects (1240-No), the electronic device may maintain displaying the visual objects based on operation 1230.

[0158] refer to Figure 12b In operation 1250, in response to receiving the second input of operation 1240 (1240-Yes), according to an embodiment, the electronic device may determine whether the visual object identified by the second input is the first visual object for selecting a sentence. When the visual object identified by the second input is not the first visual object (1250-No), in operation 1260, according to an embodiment, the electronic device may perform a function of a second visual object identified by the second input and assigned to a second visual object different from the first visual object based on the group selected by the first input. For example, the second visual object may be mapped to a function (e.g., copying and / or cropping) for extracting text represented by the group (e.g., characters and / or words) selected by the first input.

[0159] When the visual object identified by the second input is the first visual object (1250-Yes), in operation 1270, according to an embodiment, the electronic device may identify a sentence including the group identified by the first input based on the spacing between the plurality of words distinguished by the plurality of strokes and based on the time when the plurality of words are input. Fig.11For example, when a first word is recognized by a first input, in response to a second input, the electronic device can recognize the strokes included in the sentence including the first word based on the spacing between the first word and a second word different from the first word and the moment when a plurality of words including the first word and the second word are input.

[0160] refer to Figure 12b In operation 1280, according to an embodiment, the electronic device may change the display of the visual object in operation 1230 based on the recognized sentence. For example, the electronic device may stop displaying the first visual object being displayed based on operation 1230. Changing the display of the visual object in operation 1230 may include displaying a visual object indicating that a stroke included in the sentence is recognized or selected.

[0161] Fig.13 An example of a flowchart for describing an operation performed by an electronic device based on an input received from a user according to an embodiment is shown. Fig.13 Electronic devices may include Figures 1 to 11 Electronic equipment and / or Figure 12a to Figure 12b For example, Fig.13 At least one operation can be performed by Figure 3 Executed by the electronic device 101 and / or the processor 120. Fig.13 At least one operation can be combined with Fig.11 and / or Figure 12a to Figure 12b is related to at least one operation.

[0162] refer to Fig.13 , in operation 1310, according to an embodiment, the electronic device may display a plurality of characters. Fig.13 Operation 1310 of may be performed similarly to operation 1210 of Fig. 12. For example, the electronic device may display a plurality of characters distinguished by strokes included in the media content.

[0163] refer to Fig.13 In operation 1320, according to an embodiment, the electronic device may display a function list related to at least one first character in response to a first input indicating selection of at least one first character. The first input may be recognized based on a gesture related to at least one first character among the characters displayed by operation 1310. The list may include Figure 9a Menu 914.

[0164] refer to Fig.13 In operation 1330 , according to an embodiment, the electronic device may determine whether a second input based on the list is received. In response to not receiving the second input based on the list ( 1330 —No), the electronic device may maintain the display list based on operation 1320 .

[0165] In operation 1340, in response to receiving the second input (1330-Yes), according to an embodiment, the electronic device may determine whether the second input indicates execution of a first function for recognizing a sentence including at least one first character. When the second input indicates execution of a second function different from the first function (1340-No), in operation 1350, according to an embodiment, the electronic device may execute a second function corresponding to the second input and different from the first function. The second function may be related to text corresponding to the at least one first character. The second function may correspond to a text different from the first character. Figure 9a Other visual objects in the menu 914 of the visual object 915 used to identify the sentence.

[0166] When the second input indicates execution of the first function (1340-Yes), in operation 1360, according to an embodiment, the electronic device may recognize a sentence including at least one first character from a plurality of characters. The electronic device may recognize the sentence based on Figure 3 For example, the electronic device may identify other input characters that are temporally and spatially adjacent to at least one first character among the plurality of characters. The electronic device may identify other characters based on Fig.11 Operation and / or Figure 12b Operation 1270 is performed.

[0167] refer to Fig.13 In operation 1370, according to an embodiment, the electronic device may display a screen based on the recognized sentence. For example, the electronic device may visually show that the recognized sentence in the screen is selected, such as Figure 8c Status 830, Figure 9b Status 920 and / or Fig.10 The electronic device may display a visual object (e.g., Figure 8c The electronic device may display the recognized sentence in a different color from other sentences. Based on operation 1370, the electronic device may highlight multimedia content related to the recognized sentence in the media content.

[0168] As described above, according to an embodiment, an electronic device may display media content including handwriting represented by a plurality of strokes. In response to an input indicating selection of at least one stroke (e.g., an input indicating selection of a character and / or a word distinguished by at least one stroke), the electronic device may display a visual object for performing a function for identifying a sentence including at least one stroke. In response to another input indicating selection of the visual object, the electronic device may identify a sentence including at least one stroke based on time information and / or spatial information of a word distinguished by a plurality of strokes.

[0169] What may be needed is a method for recognizing one or more sentences from strokes representing handwriting.

[0170] As described above, according to an embodiment, the electronic device 101 may include a display 210 and a processor 120. The processor may be configured to display a plurality of strokes 514 and 512 in the display. The processor may be configured to display a first visual object 915 for identifying a first sentence 242 including at least one character in response to a first input indicating selection of at least one character 510 distinguished by a plurality of strokes. The processor may be configured to identify the strokes included in the first sentence in a plurality of strokes based on the spacing 613 between the first word 242-3 and the second word 242-2 including at least one character and based on the moment of inputting a plurality of words including the first word and the second word in response to a second input indicating selection of the first visual object. The processor may be configured to display a second visual object 832 corresponding to the recognized strokes in the plurality of strokes in the display based on the recognition of the strokes included in the first sentence. According to an embodiment, the electronic device may recognize one or more sentences from the strokes representing handwriting. According to an embodiment, the electronic device may provide a function of recognizing the strokes representing handwriting in units of sentences.

[0171] For example, the processor may be configured to recognize the strokes included in the first sentence based on information indicating at least one of an order in which the plurality of words are input indicated by the timing or the position of the plurality of words.

[0172] For example, the processor may be configured to obtain a set including at least one of the plurality of words and a probability that the set is a single sentence by using the information. The processor may be configured to identify one or more words included in the first sentence including the first word among the plurality of words based on the obtained probability.

[0173] For example, the processor may be configured to obtain the probability by using a model trained based on a set of strokes corresponding to a single sentence.

[0174] For example, the processor may be configured to obtain a spacing between a first word and a second word sequentially connected to the first word among a plurality of words. The processor may be configured to identify whether the second word is included in a second sentence different from the first sentence based on at least one of a difference between a first moment when the first word is input and a second moment when the second word is input or the obtained spacing.

[0175] For example, the processor may be configured to identify a spacing between the first word and the second word in the first direction based on identifying that the first word and the second word are included in a first row formed along a first direction in which characters distinguished by a plurality of strokes are arranged.

[0176] For example, the processor can be configured to identify the spacing between the first word and the second word in a second direction different from the first direction based on recognizing that the first word is included in a first row formed along a first direction in which characters distinguished by multiple strokes are arranged and the second word is included in a second row parallel to the first row.

[0177] For example, the processor may be configured to display at least one third visual object together with the first visual object responsive to the first input for performing a function associated with at least one word selected by the first input.

[0178] For example, the at least one third visual object may be configured to perform the function independently of other characters other than the at least one character in the first sentence.

[0179] For example, the processor may be configured to identify at least one image associated with the first sentence in the media content including a plurality of strokes in response to the second input 470. The processor may be configured to display a second visual object corresponding to the strokes included in the first sentence and the identified at least one image based on identifying the at least one image.

[0180] For example, the processor may be configured to identify an audio signal associated with a first sentence in the media content including a plurality of strokes in response to a second input. The processor may be configured to display a second visual object corresponding to the strokes included in the first sentence and a third visual object indicating the audio signal based on the identified audio signal.

[0181] For example, the processor may be configured to display the plurality of strokes in the display based on information including the time and location at which each of the plurality of strokes is input.

[0182] As described above, according to an embodiment, a method of an electronic device may include displaying 1310 a plurality of characters in a display of the electronic device. The method may include displaying 1320 a list of functions associated with at least one first character in response to a first input indicating selection of at least one first character among a plurality of characters. The method may include identifying 1360 a sentence including at least one first character among a plurality of characters in response to a second input indicating selection of a first function for identifying a sentence including at least one first character identified based on a list. The method may include displaying at least one visual object associated with a plurality of second characters included in the sentence and including at least one first character based on the identification of the sentence.

[0183] For example, displaying the at least one visual object may include displaying a list of functions associated with the sentence indicated by the plurality of second characters independently of the visual object for performing the first function.

[0184] For example, the display list may include at least one of the following items displayed together with the first visual object for performing the first function: a second visual object for performing a second function for copying at least one first character selected by the first input; or a third visual object for performing a third function of extracting at least one first character from a plurality of characters displayed in the display.

[0185] For example, the method may include at least temporarily storing in a memory of the electronic device at least one first character identified by the first input in response to a third input for selecting the second visual object.

[0186] For example, the recognition may include recognizing a sentence based on a spacing between a plurality of characters and a difference between times when each of the plurality of characters is input.

[0187] For example, displaying the plurality of characters may include displaying the plurality of characters distinguished by the plurality of strokes based on information indicating the plurality of strokes sequentially drawn by the user.

[0188] For example, displaying at least one visual object may include displaying at least one visual object in the media content including the plurality of characters so as to highlight at least one of an image, a video, or an audio associated with the recognized sentence.

[0189] As described above, according to an embodiment, a method of an electronic device may include displaying a plurality of strokes in a display of the electronic device. The method may include displaying a first visual object for identifying a first sentence including at least one character in response to a first input indicating selection of at least one character distinguished by a plurality of strokes. The method may include identifying the strokes included in the first sentence among the plurality of strokes based on the spacing between the first word and the second word including at least one character and based on the moment of inputting a plurality of words including the first word and the second word in response to a second input indicating selection of the first visual object. The method may include displaying a second visual object corresponding to the identified strokes among the plurality of strokes in the display based on the identification of the strokes included in the first sentence.

[0190] For example, the recognizing may include recognizing the strokes included in the first sentence based on information indicating at least one of an order in which the plurality of words are input indicated by the timings or positions of the plurality of words.

[0191] For example, the recognition may include obtaining a set including at least one of the plurality of words and a probability that the set is a single sentence by using the information. The recognition may include identifying one or more words included in the first sentence including the first word among the plurality of words based on the obtained probability.

[0192] For example, obtaining the probability may include obtaining the probability by using a model trained based on a set of strokes corresponding to a single sentence.

[0193] For example, the recognition may include obtaining a spacing between a first word and a second word sequentially connected to the first word among a plurality of words. The recognition may include identifying whether the second word is included in a second sentence different from the first sentence based on at least one of a difference between a first moment when the first word is input and a second moment when the second word is input or the obtained spacing.

[0194] For example, the recognizing may include recognizing a spacing between the first word and the second word in a first direction based on recognizing that the first word and the second word are included in a first row formed along a first direction in which characters distinguished by a plurality of strokes are arranged.

[0195] For example, recognition may include identifying the spacing between the first word and the second word in a second direction different from the first direction based on recognizing that the first word is included in a first row formed along a first direction in which characters distinguished by multiple strokes are arranged and the second word is included in a second row parallel to the first row.

[0196] For example, the method may further include displaying at least one third visual object together with the first visual object responsive to the first input for performing a function associated with at least one word selected by the first input. The at least one third visual object may be configured to perform the function independently of other characters other than the at least one character in the first sentence.

[0197] For example, displaying the second visual object may include identifying at least one image associated with the first sentence in the media content including the plurality of strokes in response to the second input. Displaying the second visual object may include displaying the second visual object corresponding to the strokes included in the first sentence and the identified at least one image based on identifying the at least one image.

[0198] For example, displaying the second visual object may include identifying an audio signal associated with the first sentence in the media content including the plurality of strokes in response to the second input. Displaying the second visual object may include displaying a second visual object corresponding to the strokes included in the first sentence and a third visual object indicating the audio signal based on identifying the audio signal.

[0199] For example, displaying the plurality of strokes may include displaying the plurality of strokes in a display based on information including a time and a location at which each of the plurality of strokes is input.

[0200] As described above, according to an embodiment, the electronic device 101 may include a display 210 and a processor 120. The processor may be configured to display a plurality of characters in the display. The processor may be configured to display a list of functions associated with at least one first character in response to a first input indicating the selection of at least one first character in a plurality of characters. The processor may be configured to identify a sentence including at least one first character of a plurality of characters in response to a second input indicating the selection of a first function for identifying a sentence including at least one first character, which is identified based on a list. The processor may be configured to display at least one visual object associated with a plurality of second characters included in the sentence and including at least one first character based on the recognition of the sentence.

[0201] For example, the processor may be configured to display a list of functions associated with the sentence indicated by the plurality of second characters independently of the visual object for performing the first function.

[0202] For example, the processor can be configured to display at least one of the following together with a first visual object for performing a first function: a second visual object for performing a second function for copying at least one first character selected by a first input; or a third visual object for performing a third function for extracting at least one first character from a plurality of characters displayed in a display.

[0203] For example, the electronic device may further include a memory 130. The processor may be configured to at least temporarily store the at least one first character identified by the first input in the memory of the electronic device in response to a third input for selecting the second visual object.

[0204] For example, the processor may be configured to recognize a sentence based on the spacing between the plurality of characters and the difference between the times when each of the plurality of characters is input.

[0205] For example, the processor may be configured to display a plurality of characters distinguished by a plurality of strokes based on information indicating a plurality of strokes sequentially drawn by a user.

[0206] For example, the processor may be configured to display at least one visual object in the media content 460 including the plurality of characters so as to highlight at least one of an image, a video, or an audio associated with the recognized sentence.

[0207] The above-mentioned apparatus may be implemented as a combination of hardware components, software components and / or hardware components and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as processors, controllers, arithmetic logic units (ALUs), digital signal processors, microcomputers, field programmable gate arrays (FPGAs), PLUs (programmable logic units), microprocessors, and any other devices capable of executing and responding to instructions. The processing device may execute an operating system OS and one or more software applications executed on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, although it may be described as using a processing device, it can be seen by those skilled in the art that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, the processing device may include multiple processors or a processor and a controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0208] Software may include computer programs, codes, instructions, or a combination of one or more of them, and configures a processing device to operate as desired or to command the processing device, either independently or in combination. Software and / or data may be embodied in any type of machine, component, physical device, computer storage medium, or device for interpreting instructions or data by or providing instructions or data to a processing device. Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

[0209] The method according to the embodiment can be implemented in the form of program instructions, which can be executed by various computer devices and recorded in computer-readable media. In this case, the medium can continuously store computer executable programs or temporarily store programs for execution or downloading. In addition, the medium can be a variety of recording devices or storage devices, in which single or multiple hardware are combined, and is not limited to media directly connected to any computer system, and can be distributed on a network. Examples of media can include magnetic media (such as hard disks, floppy disks, and tapes), optical recording media (such as CD-ROMs and DVDs), magneto-optical media (such as floppy disks, ROMs, RAMs, flash memories, etc.) to store program instructions. Examples of other media include application stores that distribute applications, sites that provide or distribute various software, and recording media or storage media managed by servers.

[0210] Although the embodiments have been described based on the above limited embodiments and drawings, it is apparent to those skilled in the art that various modifications and variations may be made based on the above description. For example, even if the techniques are performed in a different order from the described method, and / or components such as the described systems, structures, devices, circuits, etc. are combined or combined in a different form from the described method, or replaced or substituted by other components or equivalents, appropriate results may be achieved.

[0211] Accordingly, other implementations, other embodiments, and equivalents of the claims are within the scope of the claims that will be described later.

Claims

1. An electronic device (101), comprising: Display (210); as well as Processor (120), Wherein, the processor is configured to: displaying a plurality of strokes in the display (512, 514); In response to a first input indicating selection of at least one character (510) distinguished by the plurality of strokes, displaying a first visual object (915) for identifying a first sentence (242) including the at least one character; In response to a second input indicating selection of the first visual object, identifying a stroke included in the first sentence among the plurality of strokes based on a spacing (613) between a first word (242-3) and a second word (242-2) including the at least one character and based on a time at which a plurality of words including the first word and the second word are input; and Based on the recognition of the strokes included in the first sentence, a second visual object corresponding to the recognized strokes among the plurality of strokes is displayed in the display (832).

2. The electronic device according to claim 1, wherein: The processor is configured to: Based on information indicating an order in which the plurality of words are input, strokes included in the first sentence are recognized, wherein the order is indicated by the timing or positions of the plurality of words.

3. An electronic device as claimed in any preceding claim, wherein: The processor is configured to: By using the information, obtaining a set including at least one of the plurality of words and a probability that the set is a single sentence; as well as Based on the obtained probabilities, one or more words included in the first sentence including the first word are identified among the plurality of words.

4. An electronic device as claimed in any preceding claim, wherein: The processor is configured to: The probabilities are obtained by using a model trained on a set of strokes corresponding to a single sentence.

5. An electronic device as claimed in any preceding claim, wherein: The processor is configured to: Obtaining the spacing between the first word and the second word sequentially connected to the first word among the plurality of words; as well as Based on at least one of the difference between a first time when the first word is input and a second time when the second word is input or the distance, it is identified whether the second word is included in a second sentence different from the first sentence.

6. An electronic device as claimed in any preceding claim, wherein: The processor is configured to: Based on recognizing that the first word and the second word are included in a first row formed along a first direction, the interval between the first word and the second word is recognized in the first direction, and characters distinguished by the plurality of strokes are arranged in the first direction.

7. An electronic device as claimed in any preceding claim, wherein: The processor is configured to: Based on recognizing that the first word is included in a first row formed along a first direction and the second word is included in a second row parallel to the first row, another spacing between the first word and the second word is identified in a second direction different from the first direction, and the characters distinguished by the multiple strokes are arranged in the first direction.

8. An electronic device as claimed in any preceding claim, wherein: The processor is configured to: At least one third visual object is displayed along with the first visual object in response to the first input, the third visual object being operable to perform a function associated with at least one word selected by the first input.

9. An electronic device as claimed in any preceding claim, wherein: The at least one third visual object is configured as: The function is performed independently of other characters than the at least one character in the first sentence.

10. An electronic device as claimed in any preceding claim, wherein: The processor is configured to: In response to the second input, identifying at least one image associated with the first sentence in the media content including the plurality of strokes (470); and Based on recognizing the at least one image, the second visual object corresponding to the strokes included in the first sentence and the recognized at least one image are displayed.

11. An electronic device as claimed in any preceding claim, wherein: The processor is configured to: identifying, in response to the second input, an audio signal associated with the first sentence in the media content including the plurality of strokes; and Based on recognizing the audio signal, the second visual object corresponding to the strokes included in the first sentence and a third visual object indicating the audio signal are displayed.

12. An electronic device as claimed in any preceding claim, wherein: The processor is configured to: The plurality of strokes are displayed in the display based on information including a specific time and a specific position at which each of the plurality of strokes is input.

13. A method of an electronic device, comprising: Displaying (1310) a plurality of characters on a display of the electronic device; In response to a first input indicating selection of at least one first character among the plurality of characters, displaying (1320) a list of functions associated with the at least one first character; In response to a second input identified based on the list indicating selection of a first function for identifying a sentence including the at least one first character, identifying (1360) the sentence including the at least one first character among the plurality of characters; as well as Based on the recognition of the sentence, at least one visual object associated with a plurality of second characters included in the sentence and including the at least one first character is displayed.

14. The method of claim 13, wherein: Displaying the at least one visual object comprises: Independently from the first visual object for executing the first function, another function list associated with the sentence indicated by the plurality of second characters is displayed.

15. A method as claimed in any one of the preceding claims, wherein: Displaying the other list comprises: At least one of a second visual object or a third visual object is displayed together with the first visual object for performing the first function, wherein the second visual object is used to perform the second function of copying the at least one first character selected by the first input, and the third visual object is used to perform the third function of extracting the at least one first character from the multiple characters displayed in the display.

Images