Electronic device and method of processing handwritten input thereof

Abstract

An electronic device for processing handwritten input is disclosed, which includes a touch screen, a processor operatively connected with the touch screen, and a memory operatively connected with the processor, wherein the memory stores instructions which, when executed, cause the processor to control the electronic device to: perform handwriting recognition on a first handwritten input of a user displayed on the touch screen to convert the first handwritten input into text; identify at least one of attributes or features of the first handwritten input; apply at least one of the identified attributes or features to the converted text; and replace the first handwritten input with the text to which at least one of the identified attributes or features has been applied (hereinafter referred to as first rich text) in response to a request for conversion of the first handwritten input.

Description

TECHNICAL FIELD

[0001] The disclosure relates to an electronic device and a method of processing handwriting input in the electronic device. BACKGROUND

[0002] Electronic devices such as mobile terminals, smart phones, and wearable terminals can provide various functions, for example, music play, navigation, short-range wireless communication including Bluetooth, wireless fidelity (Wi-Fi), and near field communication (NFC), fingerprint recognition, photographing or video shooting, and electronic payment.

[0003] Electronic devices can also include a writing input function that uses an input device such as a handwriting pen, an electronic pen, or a user's finger, and can provide a writing recognition function that converts a writing input into text. SUMMARY

[0004] TECHNICAL PROBLEM

[0005] Through a writing input, a user can configure various attributes such as color, thickness, and writing brush type. A writing input can have various features such as letter size, line type, alignment, writing style, pressure, slope, and stroke features including the slope of the end of a stroke. However, an electronic device generally converts a writing input into text having specified attributes without considering the attributes and features of the writing input at the time of writing recognition. Therefore, the writing recognition function has problems and / or inconveniences because a user must change the attributes of the text after completing the text conversion.

[0006] The writing recognition function can be automatically executed according to a user's command after a writing input has been completed or when it is determined that the writing input has been completed, for example, when no writing input is detected for a predetermined time. Therefore, the writing recognition function is more inconvenient because a user of the electronic device cannot know the writing recognition result before the writing recognition is completed.

[0007] Therefore, there is a need in the art for an electronic device that can more conveniently enable a user to process attribute modification and determine a writing recognition result during a writing input function.

[0008] Aspects of the disclosure are directed to at least solving the above-mentioned problems and / or disadvantages and to at least providing the advantages described below. Accordingly, an aspect of the disclosure is to provide an electronic device that considers attributes and / or features of handwriting input to perform handwriting recognition.

[0009] Another aspect of the disclosure is to provide an electronic device that performs handwriting recognition on handwriting input in real time and presents a preview of a handwriting recognition result.

[0010] SOLUTION TO PROBLEM

[0011] According to an aspect of the disclosure, an electronic device can include a touch screen, a processor operatively connected with the touch screen, and a memory operatively connected with the processor, wherein the memory stores instructions that, when executed, cause the processor to control the electronic device to perform handwriting recognition on a first handwriting input of a user displayed on the touch screen to convert the first handwriting input into text, identify at least one of attributes or features of the first handwriting input, apply the identified at least one of attributes or features to the converted text, and replace the first handwriting input with the text to which the identified at least one of attributes or features has been applied (hereinafter, first rich text) in response to a request for conversion of the first handwriting input.

[0012] According to another aspect of the disclosure, a method of processing handwriting input in an electronic device includes converting a first handwriting input of a user into text by performing handwriting recognition on the first handwriting input, identifying at least one of attributes or features of the first handwriting input, applying the identified at least one of attributes or features to the converted text, and replacing the first handwriting input with the text to which the identified at least one of attributes or features has been applied (hereinafter, first rich text) in response to a request for conversion of the first handwriting input.

[0013] Advantages of the Invention

[0014] Embodiments of the disclosure can consider attributes and / or features of handwriting input at the time of handwriting recognition to perform text conversion. Embodiments of the disclosure can perform handwriting recognition in real time and present a handwriting recognition result through a preview window, so a user can recognize the handwriting recognition result in real time. In response to a problem (e.g., erroneous recognition or recognition impossibility) in handwriting recognition, the user can immediately modify the handwriting input, thereby improving the user's convenience. BRIEF DESCRIPTION OF DRAWINGS

[0015] The above and other aspects, features, and advantages of certain embodiments of the disclosure will be more apparent from the following detailed description, taken in conjunction with the accompanying drawings, in which:

[0016] Figure 1 is a block diagram of an electronic device in a network environment according to an embodiment;

[0017] Figure 2 shows a method of processing handwriting input in an electronic device according to an embodiment;

[0018] Figure 3 shows a method of processing handwriting input in an electronic device according to an embodiment;

[0019] Figure 4A and Figure 4BA method of processing a handwritten input in an electronic device according to an embodiment is shown;

[0020] Figure 5 A method of converting a handwritten input into text in an electronic device according to an embodiment is shown;

[0021] Figure 6A and Figure 6B A method of converting a handwritten input into text in an electronic device according to an embodiment is shown;

[0022] Figure 7 A method of processing a handwritten input in an electronic device according to an embodiment is shown;

[0023] Figure 8A 、 Figure 8B 、 Figure 8C 、 Figure 8D and Figure 8E A method of processing a handwritten input in an electronic device according to an embodiment is shown;

[0024] Figure 9A A method of converting a handwritten input into text in an electronic device according to an embodiment is shown;

[0025] Figure 9B A method of converting a handwritten input into text in an electronic device according to an embodiment is shown;

[0026] Figure 10 A method of adding text by using a handwritten input in an electronic device according to an embodiment is shown;

[0027] Figure 11A and Figure 11B A method of adding text by using a handwritten input in an electronic device according to an embodiment is shown;

[0028] Figure 12 A method of modifying text by using a handwritten input in an electronic device according to an embodiment is shown;

[0029] Figure 13 A method of modifying text by using a handwritten input in an electronic device according to an embodiment is shown. DETAILED DESCRIPTION

[0030] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Although specific embodiments are illustrated in the drawings and described in detail, it should be understood that the embodiments are not limited to the specific forms as disclosed. For example, it will be apparent that modifications are possible to the embodiments for a person skilled in the art to which the present disclosure pertains. Detailed descriptions of known functions and / or configurations incorporated herein will be omitted for clarity and conciseness.

[0031] Figure 1 is a block diagram illustrating an electronic device 101 in a network environment 100 according to an embodiment.

[0032] Referring to Figure 1 , the electronic device 101 can communicate with an electronic device 102 via a first network 198 (e.g., a short-range wireless communication network), or an electronic device 104 or a server 108 via a second network 199 (e.g., a long-range wireless communication network). The electronic device 101 can communicate with the electronic device 104 via the server 108. According to an embodiment, the electronic device 101 can include a processor 120, a memory 130, an input device 150, a sound output device 155, a display device 160, an audio module 170, a sensor module 176, an interface 177, a haptic module 179, a camera module 180, a power management module 188, a battery 189, a communication module 190, a subscriber identification module (SIM) card 196, or an antenna module 197. At least one (e.g., the display device 160 or the camera module 180) of the components can be omitted from the electronic device 101, or one or more other components can be added in the electronic device 101. Some of the components can be implemented as single integrated circuitry. For example, the sensor module 176 (e.g., a fingerprint sensor, an iris sensor, or an illuminance sensor) can be implemented as embedded in the display device 160 (e.g., a display).

[0033] The processor 120 can execute, for example, software (e.g., a program 140) to control at least one other component (e.g., a hardware or software component) of the electronic device 101 coupled with the processor 120, and can perform various data processing or computation. As at least part of the data processing or computation, the processor 120 can load a command or data received from another component (e.g., the sensor module 176 or the communication module 190) to a volatile memory 132, process the command or data stored in the volatile memory 132, and store resulting data in a non-volatile memory 134. The processor 120 can include a main processor 121 (e.g., a central processing unit (CPU) or an application processor (AP)), and an auxiliary processor 123 (e.g., a graphics processing unit (GPU), an image signal processor (ISP), a sensor hub processor, or a communication processor (CP)) that is operable independently from, or in conjunction with, the main processor 121. Additionally or alternatively, the auxiliary processor 123 can be adapted to consume less power than the main processor 121, or to be specific to a particular function. The auxiliary processor 123 can be implemented as a separate processor, or as part of the main processor 121.

[0034] The auxiliary processor 123 can control at least some of the functions or states related to at least one component (e.g., the display device 160, the sensor module 176, or the communication module 190) among the components of the electronic device 101, instead of the main processor 121 while the main processor 121 is in an inactive (e.g., sleep) state, or together with the main processor 121 while the main processor 121 is in an active state (e.g., executing an application), or can control at least some of the functions or states related to at least one component (e.g., the display device 160, the sensor module 176, or the communication module 190) among the components of the electronic device 101. The auxiliary processor 123 (e.g., an ISP or a CP) can be implemented as a part of another component functionally related to the auxiliary processor 123 (e.g., the camera module 180 or the communication module 190).

[0035] The memory 130 can store data used by at least one component (e.g., the processor 120 or the sensor module 176) of the electronic device 101. The data can include, for example, software (e.g., the program 140), input data or output data for commands related thereto, or the like. The memory 130 can include the volatile memory 132 or the non-volatile memory 134.

[0036] The program 140 can be stored in the memory 130 as software, and can include an operating system (OS) 142, middleware 144, or an application 146.

[0037] The input device 150 can receive a command or data, which is used for at least one component (e.g., the processor 120) of the electronic device 101, from the outside (e.g., a user) of the electronic device 101. The input device 150 can include a microphone, a mouse, a keyboard, or a digital pen (e.g., a stylus pen).

[0038] The sound output device 155 can output sound signals to the outside of the electronic device 101. The sound output device 155 can include a speaker or a receiver. The speaker can be used for general purposes, such as playing multimedia or playing record, and the receiver can be used for incoming calls. The receiver can be implemented as separate from the speaker, or can be implemented as a part of the speaker.

[0039] The display device 160 can visually provide information to the outside (e.g., a user) of the electronic device 101. The display device 160 can include a display, a hologram device, or a projector, and a control circuit for controlling a corresponding one of the display, the hologram device, and the projector. The display device 160 can include a touch circuit adapted to detect a touch, or a sensor circuit (e.g., a pressure sensor) adapted to measure the intensity of force incurred by the touch.

[0040] The audio module 170 can convert a sound into an electrical signal and vice versa. The audio module 170 can obtain the sound from a microphone of the input device 150 or output the sound through a speaker of the sound output device 155 or a headphone of an external electronic device (e.g., an electronic device 102) directly (e.g., wiredly) or wirelessly coupled with the electronic device 101.

[0041] The sensor module 176 can detect an operational state (e.g., power or temperature) of the electronic device 101 or an environmental state (e.g., a state of a user) external to the electronic device 101, and then generate an electrical signal or data value corresponding to the detected state. The sensor module 176 can include a gesture sensor, a gyro sensor, an atmospheric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0042] The interface 177 can support one or more specified protocols to be used for the electronic device 101 to be coupled with the external electronic device (e.g., the electronic device 102) directly (e.g., wiredly) or wirelessly. The interface 177 can include a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface, or an audio interface.

[0043] The connection terminal 178 can include a connector through which the electronic device 101 can be physically connected with the external electronic device (e.g., the electronic device 102). The connection terminal 178 can include a HDMI connector, a USB connector, a SD card connector, or an audio connector (e.g., a headphone connector).

[0044] The haptic module 179 can convert an electrical signal into a mechanical stimulus (e.g., a vibration or movement) or an electrical stimulus that can be recognized by a user via his tactile sensation or kinesthetic sensation. The haptic module 179 can include a motor, a piezoelectric element, or an electrical stimulus.

[0045] The camera module 180 can capture a still image or moving images. The camera module 180 can include one or more lenses, image sensors, ISPs, or flashes.

[0046] The power management module 188 can manage power supplied to the electronic device 101. The power management module 188 can be implemented as at least part of a power management integrated circuit (PMIC).

[0047] The battery 189 can supply power to at least one component of the electronic device 101. The battery 189 can include a primary cell which is not rechargeable, a secondary cell which is rechargeable, or a fuel cell.

[0048] The communication module 190 can support establishing a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (e.g., the electronic device 102, the electronic device 104, or the server 108) and performing communication between the established communication channel. The communication module 190 can include one or more CPs that are operable independently from the processor 120 (e.g., an application processor (AP)) and supports a direct (e.g., wired) communication or a wireless communication. The communication module 190 can include a wireless communication module 192 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (e.g., a local area network (LAN) communication module or a power line communication (PLC) module). A corresponding one of these communication modules can communicate with an external electronic device, via a first network 198 (e.g., a short-range communication network, such as Bluetooth, Wi-Fi direct, or infrared data association (IrDA)) or a second network 199 (e.g., a long-range communication network, such as a cellular network, the Internet, or a computer network (e.g., LAN or wide area network (WAN)). These various types of communication modules can be implemented as a single component (e.g., a single chip), or can be implemented as separate components (e.g., separate chips) from each other. The wireless communication module 192 can identify and authenticate the electronic device 101 in a communication network, such as the first network 198 or the second network 199, using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the SIM card 196.

[0049] The antenna module 197 can transmit or receive a signal or power to or from an external electronic device (e.g., an external electronic device) of the electronic device 101. The antenna module 197 can include an antenna including a radiating element composed of a conductive material or a conductive pattern formed in or on a substrate, such as a printed circuit board (PCB). The antenna module 197 can include a plurality of antennas. In this case, at least one antenna appropriate for a communication scheme used in a communication network, such as the first network 198 or the second network 199, can be selected from the plurality of antennas by the communication module 190 (e.g., the wireless communication module 192). Then, the signal or the power can be transmitted or received between the communication module 190 and an external electronic device via the selected at least one antenna. An additional component (e.g., a radio frequency integrated circuit (RFIC)) other than the radiating element can be additionally formed as part of the antenna module 197.

[0050] At least some of the above-described components can be coupled to each other via an inter-peripheral communication scheme (e.g., a bus, general purpose input output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)), and communicate information (e.g., commands or data) between them.

[0051] Commands or data can be transmitted or received between the electronic device 101 and the external electronic device 104 via the server 108 coupled with the second network 199. Each of the electronic devices 102 and 104 can be a device of a same type as, or a different type from, the electronic device 101. All or some of the operations to be executed at the electronic device 101 can be executed at one or more of the external electronic devices 102, 104, or server 108. For example, if the electronic device 101 is to perform a function or a service automatically, or in response to a request from a user or another device, the electronic device 101, instead of, or in addition to, executing the function or the service, can request at least a part of the function or the service to be executed by one or more of the external electronic devices. The one or more external electronic devices receiving the request can perform the requested at least a part of the function or the service, or an additional function or an additional service related to the request, and transfer an execution result to the electronic device 101. The electronic device 101 can process the result, or provide the result as it is, as at least partial reply to the request. To this end, for example, a cloud, distributed, or client-server computing technology can be used.

[0052] According to an embodiment, an electronic device (e.g., the electronic device 101) can include a touch screen (e.g., the display device 160), a processor (e.g., the processor 120) operatively connected with the touch screen, and a memory (e.g., the memory 130) operatively connected with the processor, wherein the memory can store instructions that, when executed, cause the processor to control the electronic device to perform handwriting recognition on a first handwriting input of a user displayed on the touch screen to convert the first handwriting input into text, identify at least one of attributes or features of the first handwriting input, apply at least one of the identified attributes or features to the converted text, and replace the first handwriting input with the text to which at least one of the identified attributes or features has been applied (hereinafter referred to as a first rich text) in response to a request for conversion of the first handwriting input.

[0053] The memory can further store instructions that cause the processor to control the electronic device to present a preview window including the first rich text to a location adjacent to the first handwriting input, before replacing the first handwriting input with the first rich text.

[0054] The memory can further store instructions that cause the processor to control the electronic device to copy the first rich text included in the preview window, in response to detecting a request to copy the first rich text, or to display the first handwriting input together with the first rich text, in response to detecting a request to obtain the first rich text.

[0055] The memory can further store instructions that cause the processor to control the electronic device to detect the first handwriting input, and to perform handwriting recognition on the first handwriting input, in response to detecting no additional handwriting input within a specified time after detecting the first handwriting input or in response to receiving a handwriting recognition command of a user.

[0056] The memory can further store instructions that cause the processor to control the electronic device to re-perform handwriting recognition for a range that has been specified with a portion of the first handwriting input, in response to editing the portion.

[0057] The preview window can include rich text of a specified range of the first handwriting input, wherein the specified range can include one line, a plurality of lines specified with a standard of a location where the handwriting input has been provided, or one screen, and can be changed by a user.

[0058] The memory can further store instructions that cause the processor to control the electronic device to identify a relationship between a second handwriting input and the first rich text, in response to adding the second handwriting input after replacing the first handwriting input with the first rich text, determine a location of a second rich text in which the second handwriting input has been converted based on the identified relationship, and display the second rich text at the determined location.

[0059] The memory can further store instructions that cause the processor to control the electronic device to replace at least a portion of the first rich text with a third rich text in which a third handwriting input has been converted, in response to adding the third handwriting input overlapping at least the portion of the first rich text after replacing the first handwriting input with the first rich text.

[0060] The memory can further store instructions that cause the processor to control the electronic device to classify the first handwriting input into text and non-text, perform line separation on the classified text, identify at least one target line on which handwriting recognition is to be performed, and perform handwriting recognition on all or a portion of the identified at least one target line.

[0061] The memory can further store instructions that cause the processor to control the electronic device to output an error message in response to being unable to perform line separation or being unable to perform handwriting recognition.

[0062] Figure 2 A method of processing handwriting input in an electronic device according to an embodiment is illustrated.

[0063] Reference Figure 2 In operation 201, a processor (e.g., the processor 120) of an electronic device (e.g., the electronic device 101) can turn on (or enable) a preview function of handwriting input. The preview function can recognize a user's handwriting input using an input device (e.g., the input device 150) (e.g., a digital pen, a stylus, a finger, etc.) and present its recognition result (e.g., text) through a preview window.

[0064] In response to execution of an application in which handwriting input can be made (hereinafter, a handwriting application) (e.g., a memo application, a note application, etc.), the processor can automatically turn on the preview function. The preview function can be turned on or off by a user through a configuration menu of the electronic device, a menu included in the handwriting application, or a designated gesture input.

[0065] In operation 203, the processor can sense handwriting input. For example, the processor can sense handwriting input through a touch circuit (e.g., a capacitive touch circuit, an electromagnetic induction touch circuit, etc.) included in a display (e.g., the display device 160).

[0066] In operation 205, the processor can perform handwriting recognition. For example, the processor can convert handwriting input into text through handwriting recognition. The text conversion method will be described in detail later with reference to Figure 5 、 Figure 6A and Figure 6B .

[0067] In operation 207, the processor can identify attributes and / or features of handwriting input. For example, the processor can identify attributes of handwriting input, such as color, thickness, and writing brush type, and / or features of handwriting input, including letter size, line, alignment, slope, and stroke features (e.g., slope of the end of a stroke).

[0068] In operation 209, the processor can apply the identified attributes and / or features to the text. For example, the processor can separate the lines of the text corresponding to the separation of the lines of the handwriting input, align the lines of the text corresponding to the alignment of the lines of the handwriting input, determine the color of the text to correspond to the color of the handwriting input, determine the size of the text based on the size of the handwriting input, determine the thickness of the text (e.g., determine to apply or not to apply bold) corresponding to the thickness (or pressure at the handwriting input) of the handwriting input, determine the slope of the text (e.g., determine to apply or not to apply a slope shape) corresponding to the slope of the handwriting input (or the slope of the electronic pen), and determine to apply or not to apply a serif font to the text according to the pen type selected at the handwriting input (or the stroke characteristics of the handwriting input).

[0069] In operation 211, the processor can display the text to which the attributes and / or features have been applied (hereinafter, rich text) on the preview window in real time. For example, the processor can display the preview window including the rich text immediately corresponding to the handwriting input or when it is determined that the handwriting input has been completed.

[0070] Figure 3 A method of processing handwriting input in an electronic device according to an embodiment is illustrated.

[0071] In Figure 3 In the screenshots 310, 320, and 330, the processor can perform handwriting recognition in real time corresponding to the sensing of the handwriting input (e.g., stroke input), display a preview window 301 at a position (e.g., above) adjacent to the handwriting input, and display the recognized rich text on the preview window 301 in real time. As Figure 3 As illustrated in the screenshot 340, in response to a portion of the handwriting input being removed, the processor can re-perform handwriting recognition or remove the text corresponding to the removed handwriting input.

[0072] The preview window 301 can be movable. For example, the processor can touch the preview window 301 and then move the preview window 301 corresponding to a drag input of the user.

[0073] The processor can display a visual element (e.g., a dotted line frame) 302 representing a region in which handwriting recognition is being performed. The visual element 302 can include items 302a and 302b for adjusting the size of the visual element 302. By using the items 302a and 302b, the user can change the size of the region in which handwriting recognition is to be performed. For example, the user can select (touch or drag) the first item 302a and expand the size of the visual element 302 upward, and can select (touch or drag) the second item 302b and expand the size of the visual element 302 downward. The size of the preview window 301 can change in response to the change in the size of the visual element 302. The visual element 302 can change in size leftward and rightward.

[0074] In response to the change in the size of the handwriting recognition area, the processor can re-perform handwriting recognition on the entire area of the change. The processor can perform handwriting recognition on the added recognition area and add the handwriting recognition result of the added recognition area to the existing handwriting recognition result. Accordingly, the processor can improve the performance (e.g., speed) of handwriting recognition (e.g., document layout analysis, line separation, and / or text recognition) of the electronic device.

[0075] Figure 4A and Figure 4B A method of processing handwriting input in an electronic device according to an embodiment is illustrated.

[0076] In response to determining that handwriting input has been completed, the processor can display a preview window. As illustrated in Figure 4A screenshot 410 and Figure 4B screenshot 450, the processor can sense handwriting input by the electronic pen 401. As illustrated in Figure 4A screenshot 420 and Figure 4B screenshot 460, in response to no additional handwriting input being sensed during a predetermined time (e.g., 300 milliseconds (ms)) after the sensing of the handwriting input, the processor can determine an area below the handwriting area in which handwriting recognition has been performed and display a visual element (e.g., a box) 402 representing the handwriting area and a preview window 403 above the handwriting area. By touching (e.g., tapping) the handwriting area, the preview window 403 can display a guide phrase 403a informing that text conversion can be made.

[0077] In response to sensing a touch in the handwriting area, as illustrated in Figure 4A screenshot 430 and Figure 4B screenshot 470, the processor can display a recognition result (e.g., rich text) 403b of the handwriting input on the preview window 403. By touching (e.g., tapping) the preview window 403, the processor can display a guide phrase 403c informing that the handwriting input can be converted into text together on the preview window 403.

[0078] Referring back to Figure 2 , in Figure 4A and Figure 4B , the processor can perform operations 205 to 209 before requesting display of the preview window 403 (e.g., before touching the handwriting area). The processor can perform some of operations 205 to 209 as a background before requesting display of the preview window 403 and perform the remaining some operations after the display of the preview window 403 has been requested. The processor can perform operations 205 to 209 after selecting the handwriting area.

[0079] Since the user has turned off the preview function, operation 211 can be omitted.

[0080] In operation 213, the processor can identify whether a conversion request has been sensed. In the example, the processor can identify whether a specified touch input (e.g., touch / long touch / double touch, etc.) has been sensed in the preview window, whether input to a specified button on the stylus has been sensed, and whether a conversion menu displayed on the side of the preview window has been selected (e.g., touched).

[0081] In response to the absence of a detected conversion request as a recognition result of operation 213, the processor may proceed to operation 217, which will be described later. However, in response to the detection of a conversion request as a recognition result of operation 213, in operation 215, the processor may convert (or replace) the handwritten input with rich text. For example, as Figure 3 As shown in screenshot 350, the processor can remove the handwritten input and display rich text 305 that has had the attributes and / or features of the handwritten input applied. Similarly, as... Figure 4A Screenshot 440 and Figure 4B As shown in screenshot 480, the processor can remove the handwriting input and display rich text 405 that has had the attributes and / or features of the handwriting input applied.

[0082] In operation 217, the processor can identify whether handwriting input, such as in a handwriting application, has ended. If the handwriting input has not ended as a result of operation 217, the processor can return to operation 203 and repeat the aforementioned operation. However, if the handwriting input has ended as a result of operation 217, the processor can terminate the handwriting input process.

[0083] Figure 5 A method for converting handwritten input into text in an electronic device according to an embodiment is shown. Figure 6A and 6B A method for converting handwritten input into text in an electronic device according to an embodiment is shown.

[0084] refer to Figure 5 , Figure 6A and Figure 6B In operation 501, the processor (e.g., processor 120) can classify text and non-text by using Document Layout Analysis (DLA) at each edit (e.g., input or removal) sensing. Classification can be performed in real-time with each stroke input. Figure 6AAs illustrated, based on a database pre-stored for text and non-text classification, the processor can classify the drawing inputs 601a and 601b as non-text, and classify the handwriting inputs 603a, 603b, and 603c as text. The database can be provided by obtaining features from non-text data samples and text data samples, or can be provided by training non-text data samples and text data samples using artificial intelligence (e.g., deep learning).

[0085] The processor can classify the text and the non-text based on a set rule, for example, a limit on a height of an input stroke and a length of a stroke.

[0086] In operation 503, the processor can perform line separation on the handwriting inputs classified into the text. For example, the processor can perform line separation based on a set rule (e.g., an order of strokes of handwriting and an input location thereof). For example, the processor can separate the handwriting inputs 603a, 603b, and 603c classified into the text of Figure 6A into mutually different lines.

[0087] Operations 501 and 503 can be operated on every stroke input of an entire page. However, in response to handwriting recognition being delayed including several pages (assuming one screen as one page), the operations 501 and 503 can be performed on a current screen (i.e., page) 611 as in a screen shot 610. Figure 6B Alternatively, as in a screen shot 620, the processor can perform the operations 501 and 503 on a predetermined range 621 (e.g., a line on which an edit has been sensed, a previous line, and a next line) with a criterion that an edited line has been sensed. The processor can identify a line range that can be affected by a sensed edit (e.g., identify a line intruded by an added stroke with a criterion of a Y-axis (i.e., a vertical direction)), and perform the operations 501 and 503 on the identified line range. Figure 6B

[0088] In operation 505, the processor can identify a target line on which an edit has been provided. For example, in response to a stroke being added (a new stroke being input, being undone after a stroke is erased, or being redone after being undone), the processor can identify a line on which the stroke has been added as a target line. In response to a stroke being erased (e.g., being undone after a stroke is input, being redone after being undone, or being erased), the processor can identify a line on which the stroke has been erased as a target line.

[0089] In operation 507, the processor can perform handwriting recognition on all or a part of the target line. The target line can be set as one line, and as referred to Figure 3 ​The aforementioned structure can be expanded into multiple lines corresponding to user input of visual elements representing the target line. In response to the inability to perform handwriting recognition, the processor can control the display to output an error message. See later for further details. Figure 9B This will be described in detail.

[0090] Figure 7 A method for processing handwritten input in an electronic device according to embodiments of the present disclosure is illustrated. Figure 8A , Figure 8B , Figure 8C , Figure 8D and Figure 8E A method for processing handwritten input in an electronic device according to embodiments of the present disclosure is illustrated.

[0091] refer to Figure 7 , Figure 8A , Figure 8B , Figure 8C , Figure 8D and Figure 8E In operation 701, the processor (e.g., processor 120) of the electronic device (e.g., electronic device 101) can display a screen that includes handwriting input. For example, as Figure 8A Screenshot 805 Figure 8B Screenshot 825 Figure 8C Screenshot 850 and Figure 8D As shown in screenshot 865, the processor can display handwriting (e.g., a memo) previously written and stored by the user on a display (e.g., display device 160). The handwriting can have various attributes and / or characteristics.

[0092] In operation 703, the processor can sense handwriting recognition requests. For example... Figure 8A Screenshot 805 and Figure 8B As shown in screenshot 825, a handwriting recognition request can be sensed via touch input (e.g., tap, double tap, long touch, etc.) 801 and 828, which represent a portion of the handwriting input (i.e., handwriting). Figure 8C As shown in screenshot 850, a handwriting recognition request can be sensed through touch input to the text conversion menu 851. Figure 8D As shown in screenshot 865, a handwriting recognition request can be sensed by touching the text acquisition menu 866. Figure 8E As shown, the text conversion menu 851 and / or the text acquisition menu 866 can be displayed around the handwriting input in a floating scheme 881.

[0093] In operation 705, the processor can perform handwriting recognition. For example, the processor can convert handwritten input into text through handwriting recognition. See later. Figure 9A Describe the text conversion method in detail.

[0094] In operation 707, the processor can identify attributes and / or features of the handwriting input. In operation 709, the processor can apply the identified attributes and / or features to the text. Operations 707 and 709 are similar to the aforementioned operations 207 and 209, and thus a detailed description thereof is omitted.

[0095] In operation 711, the processor can display the rich text to which the attributes and / or features have been applied on the preview window. As shown in screenshot 810, Figure 8A the processor can display a first preview window 811 including handwriting recognition results (e.g., rich text) adjacent to (e.g., above) the position of the line in which the user's touch input has been sensed, and display a first visual element 812 representing the handwriting recognition area. The first preview window 811 can move in the position corresponding to the user's input (e.g., touch and drag). As shown in screenshots 810 and 815, Figure 8A the handwriting recognition area can be set to one line, the criterion of which is the point at which the touch has been sensed. As shown in screenshots 810 and 815, Figure 8A the size of the handwriting recognition area can correspond to the user's input 813 adjusting the first visual element 812. The size of the first preview window 811 can also change corresponding to the change in the size of the handwriting recognition area. In response to the change in the size of the handwriting recognition area, the processor can re-perform handwriting recognition on the entire area of the change, or only on the increased recognition area.

[0096] As shown in screenshot 830, Figure 8B the processor can display a second preview window 831 including handwriting recognition results (e.g., rich text) adjacent to (e.g., above) the position of the line in which the user's touch input has been sensed, and a second visual element 832 representing the handwriting recognition area. The second preview window 831 can move in the position corresponding to the user's input (e.g., touch and drag). As shown in screenshots 830 and 835, Figure 8B the size of the handwriting recognition area can change corresponding to the user's input 833 to the second visual element 832, and the size of the second preview window 831 can also change corresponding to the change in the size of the handwriting recognition area. In response to the change in the size of the handwriting recognition area, the processor can re-perform handwriting recognition on the entire area of the change, or only on the increased recognition area.

[0097] As shown in screenshot 830, Figure 8CAs shown in screenshot 855, the processor can display a third preview window 856 above the handwriting input, including the handwriting recognition result (e.g., rich text), and a third visual element 857 representing the handwriting recognition area surrounding the handwriting input. The handwriting recognition area can be set to the entire handwriting input. The size of the handwriting recognition area can be adjusted according to the user's input to the visual element 857. In response to a change in the size of the handwriting recognition area, the processor can re-perform handwriting recognition on the entire changed area, or perform handwriting recognition only on the increased recognition area. The third preview window 856 can move within its position corresponding to the user's input (e.g., touch and drag).

[0098] like Figure 8D As shown in screenshot 870, the processor can display a fourth preview window 871, including the handwriting recognition result (e.g., rich text), in a floating scheme. The fourth preview window 871 can include the recognition result of the entire handwriting input. The position of the fourth preview window 871 can move in response to user input (e.g., touch and drag).

[0099] Return to reference Figure 7 In response to the user closing the preview function, operation 711 can be omitted.

[0100] In operation 713, the processor can identify whether a conversion, copy, or paste request has been sensed. In response to a conversion request as a result of the identification in operation 713, in operation 715, the processor can convert the handwritten input into text. For example, as... Figure 8A As shown in screenshots 815 and 820, in response to sensing a specified first touch input (e.g., a tap or double tap) 816 in the first preview window 811, the processor can convert the handwritten input into rich text with the attributes and / or features of the handwritten input already applied. The converted rich text can be displayed in each line using separate text boxes, or it can be displayed where the lines are separated in a single text box.

[0101] For example, such as Figure 8B As shown in screenshots 835 and 840, in response to sensing a specified first touch input 836 in the second preview window 831, the processor can convert the handwritten input into rich text with the handwritten input's attributes and / or features already applied. The converted rich text can be displayed in separate text boxes for each line, or it can be displayed where lines are separated within a single text box. Figure 8B As shown in screenshots 840 and 845, in response to sensing that touch input 843 has not been converted to another handwriting input 841, the processor can perform operations 705 to 711 and display a fifth preview window 846 and a fifth visual element 847.

[0102] like Figure 8C As shown in screenshots 855 and 860, in response to sensing touch input to a conversion menu 856a included in a third preview window 856, the processor can convert the handwritten input into rich text with the attributes and / or features of the handwritten input already applied. The converted rich text can be displayed in each line using separate text boxes, or it can be displayed where the lines are separated in a single text box.

[0103] In response to the copy request made as a result of the recognition in operation 713, the processor can copy the text in operation 717. In the example, in response to... Figure 8A If a specified second touch input (e.g., a long touch) is sensed in the first preview window 811, the processor can copy the rich text included in the first preview window 811. In response to... Figure 8B If a specified second touch input (e.g., a long touch) is sensed in the second preview window 831, the processor can copy the rich text included in the second preview window 831. In response to sensing an input to the rich text included in the second preview window 831... Figure 8C Touch input to the copy menu 856b in the third preview window 856 allows the processor to copy rich text included in the third preview window 856. This is in response to sensing touch input to the menu included in the third preview window 856. Figure 8D Touch input via the copy menu 871b in the fourth preview window 871 allows the processor to copy rich text included in the fourth preview window 871. The copied rich text can be pasted into the currently running handwriting application or into another application where text input is possible (e.g., a letter application, a web page, or a short message service (SMS) application).

[0104] In response to the paste request made as a recognition result of operation 713, in operation 719, the processor can paste the rich text into the specified area. For example, as Figure 8D As shown in screenshots 870 and 875, in response to sensing touch input to a paste menu 871a included in the fourth preview window 871, the processor can paste rich text with applied handwriting input attributes and / or features into input field 876 and display a virtual keyboard 877 that allows editing of the rich text included in input field 876. The pasted rich text can be displayed in separate text boxes for each line, or it can be displayed in a text box where the lines are separated. The fourth preview window 871 may include an undo menu 871c for undoing handwriting recognition and a language selection menu 871d for selecting the recognition language.

[0105] In operation 721, the processor can identify whether an end command has been sensed. In response to the absence of an end command as the result of operation 721, the processor can return to operation 703 and repeat the aforementioned operation. In response to the presence of an end command as the result of operation 721, the processor can terminate the handwriting input process.

[0106] In response to a detected request to retrieve rich text, the processor can control the display to show both the handwritten input and the rich text simultaneously, without removing the handwritten input. For example, in response to a detected request, the processor can control the display to show the rich text below the handwritten input.

[0107] Figure 9A A method for converting handwritten input into text in an electronic device according to an embodiment is shown. Figure 9B A method for converting handwritten input into text in an electronic device according to an embodiment is shown.

[0108] refer to Figure 9A and Figure 9B In operation 901, the processor may perform document layout analysis (DLA) on a designated area (e.g., a handwriting recognition area) to classify text and non-text. The designated area may include a row or a set number of rows that include points that have been sensed from the user's touch input.

[0109] In operation 903, the processor can perform line splitting on handwritten input categorized into text. For example, the processor can perform line splitting based on set rules (e.g., the stroke order of the handwriting and its input position).

[0110] In operation 905, the processor can perform handwriting recognition on a specified area. The processor can perform handwriting recognition as a background for the entire handwriting input or included in the handwriting input on the current screen. By previously performing handwriting recognition on a non-specified area, the handwriting recognition result can be quickly presented in response to the user expanding the specified area.

[0111] like Figure 9B As shown, the processor can control the display to output error message 911 in response to the inability to perform handwriting recognition. For example, the processor can obtain the slope information of the row based on known techniques (e.g., linear regression, methods for calculating the angle with the smallest aspect ratio of the corresponding row, etc.) during handwriting recognition, and control the display to output error message 911 in response to the slope information being greater than a specified angle. Alternatively, in response to the row separation failure in operation 903, the processor can control the display to output error message 911.

[0112] Figure 10 A method for adding text in an electronic device by using handwriting input is shown according to an embodiment of the present disclosure. Figure 11A andFigure 11B A method of adding text by using handwriting input in an electronic device according to an embodiment of the present disclosure is illustrated.

[0113] Referring to Figure 10 , Figure 11A and Figure 11B , in operation 1001, the processor (for example, the processor 120) can identify whether handwriting input is added after operation 215. For example, as shown in screenshots 1110, 1120, and 1130, when the first handwriting input "loved" 1111 has completed conversion to the first text 1121, the user can perform additional handwriting input (hereinafter, second handwriting input) "We met" 1131 by using the electronic pen 1101. Figure 11A

[0114] In response to no handwriting input being added as the identification result of operation 1001, the processor can proceed to operation 217, where the method ends. However, in response to handwriting input being added as the identification result of operation 1001, in operation 1003, the processor can perform handwriting recognition on the added handwriting input. For example, in response to the second handwriting input 1131 being added, the processor can display the recognition result of the second handwriting input 1131 on the preview window 1132.

[0115] In operation 1005, the processor can identify a relationship with previously converted text. For example, the processor can identify whether the first text 1121 and the second handwriting input 1131 are consecutive texts. The processor can identify the relationship based on the size, color, or position of the second handwriting input 1131. For example, as shown in screenshot 1130, in response to the size and color of the second handwriting input 1131 being similar to the size and color of the first text 1121 and the second handwriting input 1131 being initiated near the last letter of the first text 1121, the processor can determine that the inputs are related to each other (for example, consecutive sentences). Figure 11A Figure 11B For example, as shown in screenshot 1150, in response to the size and color of the third handwriting input 1151 being different from the size and color of the first text 1121 and the interval from the first text 1121 being greater than a predetermined distance, the processor can determine that the inputs are not related to each other (for example, separate sentences). The processor can identify the relationship through contextual recognition, and can determine whether to split lines or not based on whether the added handwriting input overlaps the first text 1121 in the X-axis and / or Y-axis.

[0116] ​​In operation 1007, the processor can determine a display location of the additional rich text to which the added handwriting input has been converted, based on the recognition result. In operation 1009, the processor can display the additional rich text at the determined location. For example, as shown in a screenshot 1140 of FIG. 11, Figure 11A the processor can display second text 1141 to which the second handwriting input 1131 has been converted, as a continuation of the first text 1121. The first text 1121 and the second text 1141 can be included in one text box. As shown in a screenshot 1160 of FIG. 11, Figure 11B the processor can display third text 1161 to which the third handwriting input 1151 has been converted, in a different line from the first text 1121. The first text 1121 and the third text 1161 can be included in one text box, and the lines can be separated. The first text 1121 and the third text 1161 can be included in separate text boxes.

[0117] Figure 12 A method of modifying text by using handwriting input in an electronic device according to an embodiment is illustrated. Figure 13 A method of modifying text by using handwriting input in an electronic device according to an embodiment is illustrated.

[0118] Referring to Figure 12 and Figure 13 , in operation 1201, the processor (e.g., the processor 120) can identify whether handwriting input (i.e., fourth handwriting input) overlapping a portion of the text is sensed after operation 215.

[0119] In response to the handwriting input not being sensed as the recognition result of operation 1201, the processor can proceed to operation 217, where the method ends. However, in response to the handwriting input being sensed as the recognition result of operation 1201, in operation 1203, the processor can perform handwriting recognition on the overlapping handwriting input. For example, as shown in a screenshot 1310 of FIG. 13, Figure 13 in response to the user inputting handwriting input "West" to overlap with the portion "East" of the text intended for revision, the processor can perform operations 205 to 211 of FIG. 2 and display the rich text "West" on a preview window 1311. Figure 2

[0120] In operation 1205, the processor can change a portion of the rich text to the recognized rich text. For example, as shown in a screenshot 1320 of FIG. 13, Figure 13 ​As shown in a screenshot 1320, the processor can change the portion of the text overlapping the handwriting input "East" to a new rich text "West". The change of the text can be performed at a time of a conversion request (e.g., a touch input on a preview window, an elapse of a predetermined time, a menu input, or a designated button input of an electronic pen, etc.).

[0121] According to an embodiment, a method of processing handwriting input in an electronic device (e.g., the electronic device 101) includes converting a first handwriting input of a user into text by performing handwriting recognition on the first handwriting input, recognizing at least one of attributes or features of the first handwriting input, applying at least one of the recognized attributes or features to the converted text, and replacing the first handwriting input with the text (hereinafter referred to as a first rich text) to which at least one of the recognized attributes or features has been applied, in response to a request for conversion of the first handwriting input.

[0122] The method further includes presenting a preview window including the first rich text to a location adjacent to the first handwriting input, before replacing the first handwriting input with the first rich text.

[0123] The method further includes one of: copying the first rich text included in the preview window, in response to detecting a request to copy the first rich text; or displaying the first handwriting input with the first rich text, in response to detecting a request to acquire the first rich text.

[0124] Performing handwriting recognition can include one of: performing handwriting recognition of the first handwriting input, in response to detecting the first handwriting input; performing handwriting recognition of the first handwriting input, in response to detecting no additional handwriting input within a specified time after detecting the first handwriting input; or performing handwriting recognition of the first handwriting input, in response to receiving a handwriting recognition command of the user.

[0125] The method further includes re-performing handwriting recognition with respect to a range specified by a standard of an edited portion of the first handwriting input, in response to detecting the edit of the portion of the first handwriting input.

[0126] Presenting the preview window can include presenting a rich text of a specified range of the first handwriting input through the preview window, wherein the specified range can include one line, a plurality of lines specified by a standard of a location where the handwriting input has been provided, or one screen, and can be changed by the user.

[0127] The method further includes detecting addition of a second handwriting input after replacing the first handwriting input with the first rich text, recognizing a relationship between the second handwriting input and the first rich text, determining a location of a second rich text in which the second handwriting input has been converted based on the recognized relationship, and displaying the second rich text at the determined location.

[0128] The method further includes, after replacing the first handwritten input with the first rich text, detecting an addition of a third handwritten input overlapping at least a portion of the first rich text, and replacing at least a portion of the first rich text with a third rich text into which the third handwritten input has been converted.

[0129] Performing the handwriting recognition can include classifying the first handwritten input into text and non-text, performing line separation on the classified text, identifying at least one target line on which the handwriting recognition is to be performed, and performing the handwriting recognition on all or a portion of the identified at least one target line.

[0130] The method further includes, in response to being unable to perform the line separation or being unable to perform the handwriting recognition, outputting an error message.

[0131] Embodiments of the disclosure can consider attributes and / or features of the handwriting input at the time of handwriting recognition to perform text conversion. Embodiments of the disclosure can perform handwriting recognition in real time and present the handwriting recognition result through a preview window, so the user can recognize the handwriting recognition result in real time. In response to a problem (e.g., an error recognition or an unrecognizability) existing in the handwriting recognition, the user can immediately modify the handwriting input, thereby improving the convenience of the user.

[0132] The electronic device according to an embodiment can be one of various types of electronic devices. The electronic devices can include a portable communication device (e.g., a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance. The electronic devices are not limited to those described above.

[0133] It should be understood that the embodiments of the disclosure and the terms used therein are not intended to limit the technical features set forth herein to the specific embodiments and include various changes, equivalents, or replacements for a corresponding technical feature. The same or similar components can be named by the same reference numerals, and a repeated description of which can be omitted. The terms "comprise" and "have" used in the disclosure can include one or more of the stated elements without excluding other elements. In addition, the use of terms "A or B" can mean "A or B or both A and B" unless otherwise defined in the context.

[0134] It will be understood that, if a first element (e.g., a first device) is referred to as being "operatively or communicatively coupled with" or "connected to" a second element (e.g., a second device), it holds true that the first element can be directly connected to the second element in a wired manner (e.g., electrically connected to the second element), wirelessly connected to the second element, or connected to the second element via a third element.

[0135] As used herein, the term "module" can include a unit implemented in hardware, software, or firmware, and can interchangeably be used with other terms, e.g., "logic", "logic block", "part", or "circuitry". A module can be a single integral component, or a minimum unit or part thereof, adapted to perform one or more functions. A module can be implemented in a form of an application-specific integrated circuit (ASIC).

[0136] Embodiments as set forth herein can be implemented as software (e.g., a program 140) including one or more instructions that are stored in a storage medium (e.g., internal memory 136 or external memory 138) that are readable by a machine (e.g., electronic device 101). For example, a processor (e.g., processor 120) of the machine (e.g., electronic device 101) can invoke at least one of the one or more instructions stored in the storage medium, and execute it, with or without using one or more other components under the control of the processor. This allows the machine to be operated to perform at least one function according to the at least one instruction invoked. The one or more instructions can include a code generated by a complier or a code executable by an interpreter. The machine-readable storage medium can be provided in the form of a non-transitory storage medium. Wherein, the term "non-transitory" indicates that the storage medium is a tangible device, and does not include a signal (e.g., an electromagnetic wave), but this term does not differentiate between where data is semi-permanently stored in the storage medium and where the data is temporarily stored in the storage medium.

[0137] The method according to an embodiment of the disclosure can be included and provided in a computer program product. The computer program product can be traded as a product between a seller and a buyer. The computer program product can be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or be distributed online via an application store (e.g., PlayStore TM) online (e.g., download or upload) the computer program product, or can be distributed (e.g., downloaded or uploaded) directly between two user devices (e.g., smartphones). If online distribution, at least a portion of the computer program product can be generated temporarily, or at least a portion of the computer program product can be stored temporarily in a machine-readable storage medium such as a server of a manufacturer, a server of an application store, or a storage of a relay server.

[0138] Each of the above-described components (e.g., modules or programs) can include a single entity or multiple entities. One or more of the above-described components can be omitted, or one or more other components can be added. Alternatively or additionally, a plurality of components (e.g., modules or programs) can be integrated into a single component. The integrated component can still perform one or more functions of the plurality of components before integration. Operations performed by a module, a program, or another component can be performed sequentially, in parallel, repeatedly, or in a heuristic method. One or more of the operations can be performed in a different order or omitted, or one or more other operations can be added.

[0139] While the disclosure has been particularly shown and described with reference to certain embodiments thereof, it will be understood by those skilled in the art that various changes in form and details can be made therein without departing from the scope of the subject matter as defined by the appended claims and their equivalents.

Claims

1. An electronic device, the electronic device comprising: touchscreen; processor; as well as The memory stores instructions that, when executed by the processor, cause the electronic device to: Handwriting input is displayed on the touchscreen; Detect handwriting recognition requests in response to the handwriting input; In response to the detection of the handwriting recognition request, handwriting recognition is performed on the handwriting input to recognize the text corresponding to the handwriting input; Based on the handwriting recognition, at least one of the following is identified in the handwriting input: color, size, thickness, or line separation: Apply at least one of the identified color, size, thickness, or line separation to the text; Rich text, which has been applied to the text by at least one of the recognized color, size, thickness, or line separation, is presented to a position adjacent to the handwritten input. This rich text is displayed simultaneously with the handwritten input in a separate text box, distinct from the text box containing the handwritten input. In response to a request to replace the handwritten input with the rich text, the handwritten input is replaced with the rich text. Wherein, when the instruction is executed by the processor, the electronic device: Based on the detection of the handwriting recognition request, visual elements representing the handwriting recognition area are displayed, the visual elements including items for adjusting the size of the visual elements; In response to detecting user input on the item of the visual element, the size of the handwriting recognition area is adjusted to correspond to the user input; and In response to adjusting the size of the handwriting recognition area, the rendering range of the rich text is modified by re-performing the handwriting recognition on the entire area of ​​the adjusted handwriting recognition area or by performing the handwriting recognition on the added handwriting recognition area.

2. The electronic device according to claim 1, wherein, When the instruction is executed by the processor, the electronic device: In response to a detected request to copy the rich text, copy the rich text contained in the separate text box, or In response to the detection of a request to add the rich text, the handwritten input with the rich text is displayed.

3. The electronic device according to claim 1, wherein, When the instruction is executed by the processor, the electronic device: In response to editing a portion of the handwritten input, the handwriting recognition is re-performed for the range already specified by the standard for that portion.

4. The electronic device according to claim 1, wherein, When the instruction is executed by the processor, the electronic device: The handwriting input is categorized into text and non-text. Perform line splitting on the categorized text; The identification will perform at least one target line for the handwriting recognition. Perform the handwriting recognition on all or part of at least one identified target line; as well as In response to the inability to perform the line separation or the inability to perform the handwriting recognition, an error message is output.

5. A method for processing handwritten input in an electronic device, the method comprising: Handwriting input is displayed on the touchscreen of the electronic device; Detect handwriting recognition requests in response to the handwriting input; In response to detecting the handwriting recognition request, handwriting recognition is performed on the handwriting input to identify the text corresponding to the handwriting input; Based on the handwriting recognition, at least one of the following is identified in the handwriting input: color, size, thickness, or line separation: Apply at least one of the identified color, size, thickness, or line separation to the text; Rich text, which has been applied to the text by at least one of the recognized color, size, thickness, or line separation, is presented to a position adjacent to the handwritten input. This rich text is displayed simultaneously with the handwritten input in a separate text box, distinct from the text box containing the handwritten input. In response to a request to replace the handwritten input with the rich text, the handwritten input is replaced with the rich text. The method further includes: Based on the detection of the handwriting recognition request, visual elements representing the handwriting recognition area are displayed, wherein the visual elements include items for adjusting the size of the visual elements; In response to detecting user input on the item of the visual element, the size of the handwriting recognition area is adjusted to correspond to the user input; and In response to adjusting the size of the handwriting recognition area, the rendering range of the rich text is modified by re-performing the handwriting recognition on the entire area of ​​the adjusted handwriting recognition area or by performing the handwriting recognition on the added handwriting recognition area.

6. The method of claim 5, further comprising at least one of the following: In response to a detected request to copy the rich text, copy the rich text contained in the separate text box, or In response to the detection of a request to add the rich text, the handwritten input with the rich text is displayed.

7. The method according to claim 5, further comprising: In response to detecting an edit to a portion of the handwritten input, the handwriting recognition is re-performed for the range specified by the standard using the edited portion.

8. The method according to claim 5, wherein, Performing the handwriting recognition includes: The handwriting input is categorized into text and non-text. Perform line splitting on the categorized text; The identification will perform at least one target line for the handwriting recognition. Perform the handwriting recognition on all or part of at least one identified target line; and In response to the inability to perform the line separation or the inability to perform the handwriting recognition, an error message is output.

Images