Electronic device for transforming handwriting input, operating method thereof, and recording medium

Abstract

An electronic device comprising a touch screen, at least one processor and a memory that stores instructions is provided. When executed by the at least one processor, the instructions cause the electronic device to: acquire first handwriting through the touch screen; acquire first coordinate information of points of a first plurality of strokes included in the first handwriting; acquire pen pressure information about the points of the first plurality of strokes; and provide the first coordinate information and the pen pressure information to a generative AI model so as to acquire second handwriting transformed from the first handwriting, wherein a first stroke and a second stroke connected to each other in the first handwriting are separated from the second handwriting, and the second handwriting can be displayed on the touch screen.

Description

Electronic device for transforming handwriting input, method of operation thereof and recording medium

[0001] Embodiments of the present disclosure relate to an electronic device for transforming handwriting input, a method of operating the same, and a recording medium.

[0002] User equipment such as smartphones, tablets, and wearable devices may provide various functions such as music playback, navigation, short-range wireless communication (e.g., Bluetooth, Wi-Fi, or near field communication (NFC)), fingerprint recognition, photo (still image) or video recording, or electronic payment functions.

[0003] An electronic device can detect handwriting input using an input device having a pen function (referred to herein as a "stylus pen" for convenience of explanation). The input method using a stylus pen allows for more precise touch input than the touch input method using a finger, and thus can be more usefully applied when performing applications such as note-taking or sketching.

[0004] According to one embodiment, an electronic device may include a touch screen, at least one processor, and a memory storing instructions. According to one embodiment, the instructions, when executed by the at least one processor, may cause the electronic device to obtain a first handwriting through the touch screen. According to one embodiment, the instructions, when executed by the at least one processor, may cause the electronic device to obtain first coordinate information of points of a first plurality of strokes included in the first handwriting. According to one embodiment, the instructions, when executed by the at least one processor, may cause the electronic device to obtain pressure information for the points of the first plurality of strokes. According to one embodiment, the instructions, when executed by the at least one processor, cause the electronic device to provide the first coordinate information and the pressure information to a generative AI model (360) to obtain a second handwriting transformed from the first handwriting, wherein a first stroke and a second stroke that are connected to each other in the first handwriting can be separated from the second handwriting. According to one embodiment, the instructions, when executed by the at least one processor, cause the electronic device to display the second handwriting on the touch screen.

[0005] The instructions, when executed by the at least one processor, may cause the electronic device to obtain stroke speed information of the first plurality of strokes based on distances between at least some of the points of the first plurality of strokes. The instructions, when executed by the at least one processor, may cause the electronic device to provide the stroke speed information to the generative AI model.

[0006] The instructions, when executed by the at least one processor, may cause the electronic device to acquire the second stroke if the pressure identified in the second stroke overlapping at least a portion of the first stroke is less than a specified pressure.

[0007] The instructions, when executed by the at least one processor, may cause the electronic device to acquire the second stroke if the identified stroke speed in a portion where the second stroke overlaps at least a portion of the first stroke is greater than a specified speed.

[0008] The electronic device may further include a communication circuit. The instructions, when executed by the at least one processor, may cause the electronic device to obtain, through the communication circuit, the pressure information of the points of the first plurality of strokes sensed by a sensor included in the stylus pen and transmitted by the stylus pen.

[0009] The instructions, when executed by the at least one processor, may cause the electronic device to check coordinates of the points of the first plurality of strokes at specified intervals. The instructions, when executed by the at least one processor, may cause the electronic device to obtain the stroke speed information of the first plurality of strokes based on a distance between at least some of the coordinates.

[0010] The instructions, when executed by the at least one processor, may cause the electronic device to obtain, through the touch screen, a third non-text handwriting that at least partially overlaps the first handwriting. The instructions, when executed by the at least one processor, may cause the electronic device to obtain second coordinate information of at least one stroke included in the third handwriting. The instructions, when executed by the at least one processor, may cause the electronic device to provide the second coordinate information to the generative AI model to obtain a fourth handwriting that is a modification of the third handwriting. The at least one stroke included in the third handwriting may be separated from the first plurality of strokes included in the first handwriting. The instructions, when executed by the at least one processor, may cause the electronic device to display the second handwriting and the fourth handwriting on the touch screen.

[0011] The instructions, when executed by the at least one processor, may cause the electronic device to provide the first coordinate information, the pen pressure information, and the pen speed information to the generative AI model based on determining that the first handwriting is text. The instructions, when executed by the at least one processor, may cause the electronic device to provide the second coordinate information to the generative AI model based on determining that the third handwriting is non-text.

[0012] The instructions, when executed by the at least one processor, may cause the electronic device to distinguish the first handwriting into word units. The instructions, when executed by the at least one processor, may cause the electronic device to provide information about the first handwriting distinguished into word units to the generative AI model.

[0013] The instructions, when executed by the at least one processor, may cause the electronic device to distinguish the first handwriting on a line-by-line basis. The instructions, when executed by the at least one processor, may cause the electronic device to provide the first handwriting to the generative AI model stored in the memory to obtain the second handwriting.

[0014] According to one embodiment, a method of operating an electronic device may include an operation of obtaining a first handwriting through a touch screen included in the electronic device. According to one embodiment, the method of operating the electronic device may include an operation of obtaining first coordinate information of points of a first plurality of strokes included in the first handwriting. According to one embodiment, the method of operating the electronic device may include an operation of obtaining pressure information for the points of the first plurality of strokes. According to one embodiment, the method of operating the electronic device may include an operation of providing the first coordinate information and the pressure information to a generative AI model (360) to obtain a second handwriting modified from the first handwriting, wherein a first stroke and a second stroke that are connected to each other in the first handwriting may be separated from the second handwriting. According to one embodiment, the method of operating the electronic device may include an operation of displaying the second handwriting on the touch screen.

[0015] According to one embodiment, a non-transitory computer-readable storage medium storing instructions may cause the instructions, when executed by at least one processor of an electronic device, to cause the electronic device to perform operations. The operations may include obtaining a first handwriting through a touch screen included in the electronic device. According to one embodiment, the operations may include obtaining first coordinate information of points of a first plurality of strokes included in the first handwriting. According to one embodiment, the operations may include obtaining pen pressure information for the points of the first plurality of strokes. According to one embodiment, the operations may include providing the first coordinate information and the pen pressure information to a generative AI model to obtain a second handwriting modified from the first handwriting, wherein a first stroke and a second stroke that are connected to each other in the first handwriting may be separated from the second handwriting. According to one embodiment, the operations may include displaying the second handwriting on the touch screen.

[0016] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0017] FIG. 2 is a schematic block diagram of an electronic device according to one embodiment.

[0018] FIG. 3 is a schematic block diagram of a memory according to one embodiment.

[0019] FIG. 4 is a flowchart illustrating an operation of an electronic device according to one embodiment of the present invention to transform a handwriting input representing text.

[0020] FIG. 5A is a flowchart illustrating an operation of an electronic device according to one embodiment to transform handwriting input based on pressure.

[0021] FIG. 5b is a flowchart illustrating an operation of an electronic device according to one embodiment to transform handwriting input based on a pen speed.

[0022] FIG. 6 is a flowchart illustrating an operation of an electronic device according to one embodiment of the present invention to transform a handwriting input representing non-text.

[0023] FIG. 7 is a diagram illustrating an operation of an electronic device according to one embodiment to distinguish a first handwriting input by word and line.

[0024] FIG. 8 is a drawing illustrating a portion of a first handwriting input to explain an operation of an electronic device according to one embodiment of the present invention to obtain handwriting speed information of a first handwriting input.

[0025] FIG. 9 is a drawing for explaining a portion in which a plurality of strokes included in a first handwriting input are connected by an electronic device according to one embodiment.

[0026] FIG. 10 is a diagram for explaining an operation of an electronic device according to one embodiment of the present invention to obtain a second handwriting input that is a modified first handwriting input.

[0027] FIG. 11 is a diagram illustrating an electronic device according to one embodiment of the present invention, wherein the electronic device illustrates a first handwriting input and a third handwriting input overlapping at least a portion of the first handwriting input.

[0028] FIG. 12 is a diagram for explaining an operation of an electronic device according to one embodiment of the present invention to obtain a fourth handwriting input that is a modified third handwriting input.

[0029] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with the electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of the electronic device (104) or the server (108) via a second network (199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0030] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0031] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, in the electronic device (101) itself where artificial intelligence is performed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0032] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0033] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0034] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0035] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0036] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0037] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0038] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0039] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0040] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0041] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0042] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0043] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0044] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0045] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0046] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0047] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas by, for example, the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0048] In one embodiment, the antenna module (197) may generate a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.

[0049] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0050] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0051] FIG. 2 is a schematic block diagram of an electronic device according to one embodiment.

[0052] Referring to FIG. 2, according to one embodiment, an electronic device (201) (e.g., the electronic device (101) of FIG. 1) may include a processor (220) (e.g., the processor (120) of FIG. 1), a memory (230) (e.g., the memory (130) of FIG. 1), a touch screen (260) (e.g., the display (160) of FIG. 1), and a communication circuit (290) (e.g., the communication module (190) of FIG. 1). According to one embodiment, the electronic device (201) may be implemented in the same or similar manner as the electronic device (101) of FIG. 1.

[0053] According to one embodiment, the processor (220) can control the overall operation of the electronic device (201). According to one embodiment, the processor (220) can be implemented identically or similarly to the processor (120) of FIG. 1.

[0054] According to one embodiment, the memory (230) may store instructions that cause the electronic device (201) to perform operations.

[0055] According to one embodiment, the memory (230) may store a generative artificial intelligence (AI) model (360 of FIG. 3). According to one embodiment, the generative AI model (360) may be stored in a separate server or external device. For example, the generative AI model (360) may use existing content, such as text, audio, and / or images, to newly generate content similar to the existing content. For example, the generative AI model (360) may learn patterns of content and generate new content as an inference result. For example, the generative AI model (360) may perform at least one of an in-painting operation or an out-painting operation on an image to generate (or obtain, output) a new image. Alternatively, the generative AI model (360) may receive an image corresponding to a user's handwriting input and generate (or obtain, output) a new image that has at least a partially modified shape of the handwriting input.

[0056] According to one embodiment, the processor (220) may output or display a handwriting input (e.g., a second handwriting input) that is a modified version of a handwriting input (e.g., a first handwriting input) received from a user through a touch screen (260) using a generative AI model (360) stored in a memory (230).

[0057] According to one embodiment, the processor (220) can obtain a first handwriting input representing text through the touch screen (260). According to one embodiment, the processor (220) can obtain a third handwriting input representing non-text through the touch screen (260). For example, the handwriting input can be obtained by a stylus pen.

[0058] In one embodiment, text may include sentences, words, syllables, consonants, or vowels. In one embodiment, non-text may include handwritten input other than text. For example, non-text may include shapes. However, this is merely an example, and text and non-text may not be limited thereto.

[0059] According to one embodiment, the handwriting input may include a stroke. According to one embodiment, the stroke may represent a continuous stroke from the point where the handwriting input starts to the point where the handwriting input is released. According to one embodiment, the stroke may include at least one point. According to one embodiment, the processor (220) may acquire a point at a specified time interval at a location where the stylus pen and the touch screen (260) come into contact. According to one embodiment, the processor (220) may acquire information about the order and coordinates in which at least one point included in the stroke is acquired.

[0060] According to one embodiment, when a plurality of strokes included in a first handwriting input are connected to each other or when the first handwriting input and the third handwriting input overlap each other, the processor (220) may obtain a handwriting input that is a modified version of at least one of the first handwriting input representing text or the third handwriting input representing non-text by using a generative AI model (360) stored in the memory (230). According to one embodiment, the processor (220) may determine a portion where the plurality of strokes are connected to each other based on pen pressure information and pen speed information of the plurality of strokes included in the first handwriting input by using the generative AI model (360). For example, a portion where the plurality of strokes are connected may have a relatively low pen pressure and a relatively high pen speed. According to one embodiment, the processor (220) may determine whether the first handwriting input and the third handwriting input overlap each other based on coordinate information of the first handwriting input and coordinate information of the third handwriting input by using the generative AI model (360). According to one embodiment, the processor (220) may transform the first handwriting input using the generative AI model (360) so that a portion where a plurality of strokes included in the first handwriting input are connected to each other is separated. According to one embodiment, the processor (220) may transform the third handwriting input using the generative AI model (360) so that a portion where a plurality of strokes included in the first handwriting input and at least one stroke included in the third handwriting input are connected to each other is separated. Through this, the electronic device (201) according to one embodiment may transform handwriting inputs written in an overlapping manner to be separated from each other, thereby providing a readable handwriting input to a user.

[0061] Below, according to one embodiment, the processor (220) specifically describes an operation of transforming a first handwriting input representing text using a generative AI model (360).

[0062] According to one embodiment, the processor (220) can determine whether the first handwriting input is text. For example, the first handwriting input may include various forms of handwriting (or handwriting input) recognized as text by the processor (220) or an artificial intelligence model. For example, the processor (220) can determine whether the first handwriting input is text using the artificial intelligence model stored in the memory (230). The artificial intelligence model may be implemented as a document layout analyzer. However, this is merely an example, and the processor (220) may also determine whether the first handwriting input is text using a separate algorithm.

[0063] According to one embodiment, if the processor (220) determines that the first handwriting input is text, the processor (220) may obtain coordinate information of points of the first plurality of strokes included in the first handwriting input at designated intervals. For example, the processor (220) may obtain coordinate information of points of the first plurality of strokes through a sensor.

[0064] According to one embodiment, when the processor (220) determines that the first handwriting input is text, the processor (220) may obtain pressure information identified at points of the first plurality of strokes included in the first handwriting input. According to one embodiment, the processor (220) may obtain pressure information at points of the first plurality of strokes sensed by a sensor included in the stylus pen through the communication circuit (290). Depending on the implementation, the pressure information at points of the first plurality of strokes may also be obtained by a pressure sensor included in the touch screen (260).

[0065] According to one embodiment, when the processor (220) determines that the first handwriting input is text, the processor (220) may obtain (calculate or operate) handwriting speed information of the first plurality of strokes. According to one embodiment, the processor (220) may obtain the distance between coordinates of points of the first plurality of strokes obtained at a specified time interval. Based on the distance between two adjacent coordinates of points of the first plurality of strokes and the specified time interval, the processor (220) may obtain (calculate or operate) handwriting speed information of each of the strokes connecting the points.

[0066] For example, the processor (220) can obtain the coordinates of a first point among the plurality of points included in the first plurality of strokes at a first time. The processor (220) can obtain the coordinates of a second point among the plurality of points included in the first plurality of strokes, which is obtained after the first point, at a second time after a specified time from the first time. The processor (220) can obtain (calculate, or operate) stroke speed information connecting the first point and the second point based on the coordinates of the first point, the coordinates of the second point, and the specified time. For example, the specified time may be set by a user or automatically set by the processor (220).

[0067] According to one embodiment, the processor (220) may input first coordinate information, pen pressure information, and pen speed information into the generative AI model (360). According to one embodiment, the processor (220) may input the first handwriting input into the generative AI model (360) instead of the first coordinate information, thereby obtaining a second handwriting input that is a modified version of the first handwriting input.

[0068] In addition, according to one embodiment, if the processor (220) determines that the first handwriting input is text, the processor (220) may distinguish the first handwriting input on a line-by-line basis. According to one embodiment, the processor (220) may obtain information about the first handwriting input distinguished on a line-by-line basis. For example, the processor (220) may distinguish the first handwriting input on a line-by-line basis using an artificial intelligence model stored in the memory (230). The artificial intelligence model may be implemented as a document layout analyzer. However, this is an example, and the processor (220) may distinguish the first handwriting input on a line-by-line basis using a separate algorithm. According to one embodiment, the processor (220) may further input information about the first handwriting input distinguished on a line-by-line basis to the generative AI model (360).

[0069] In addition, according to one embodiment, if the processor (220) determines that the first handwriting input is text, the processor (220) may distinguish the first handwriting input into units. According to one embodiment, the processor (220) may obtain information about the first handwriting input distinguished into words. For example, the processor (220) may distinguish the first handwriting input into words using an artificial intelligence model stored in the memory (230). The artificial intelligence model may be implemented as a document layout analyzer. However, this is an example, and the processor (220) may distinguish the first handwriting input into words using a separate algorithm. For example, the processor (220) may distinguish the first handwriting input into words based on spacing (or spacing spaces). According to one embodiment, the processor (220) may further input information about the first handwriting input distinguished into words into the generative AI model (360).

[0070] According to one embodiment, if the pressure identified at at least one point connecting the first stroke and the second stroke is determined to be less than a specified pressure, the generative AI model (360) may generate a second handwriting input (second handwriting) that is a modified version of the first handwriting input such that the first stroke corresponding to the first text and the second stroke corresponding to the second text among the first plurality of strokes are separated. According to one embodiment, the specified pressure may be automatically set by the processor (220) or may be set by the user. For example, the specified pressure may represent a pressure determined to be connected between the first stroke and the second stroke. For example, the generative AI model (360) may delete a stroke corresponding to a portion where the first stroke and the second stroke are connected. For example, the generative AI model (360) may also adjust the size of a stroke corresponding to a portion where the first stroke and the second stroke are connected to be smaller.

[0071] According to one embodiment, the generative AI model (360) may generate a second handwriting input that is modified from the first handwriting input so that the first stroke corresponding to the first text and the second stroke corresponding to the second text among the first plurality of strokes are separated, if the determined writing speed based on the distance between the points connecting the first stroke and the second stroke is greater than the designated speed. According to one embodiment, the designated speed may be automatically set by the processor (220) or may be set by the user. For example, the designated speed may represent a pressure determined to be connected between the first stroke and the second stroke. The points connecting the first stroke and the second stroke may represent points included in a portion where the first stroke and the second stroke overlap each other.

[0072] Below, according to one embodiment, the processor (220) specifically describes an operation of transforming a third handwriting input representing non-text using a generative AI model (360).

[0073] In one embodiment, the processor (220) may determine whether the third handwriting input is non-text. For example, the third handwriting input may include various forms of handwriting (or handwriting input) that are recognized as non-text by the processor (220) or an artificial intelligence model. For example, non-text may include various forms of input (e.g., handwriting input) that are not recognized as text. For example, non-text may include shapes such as continuous or discontinuous shapes, lines, points, and / or curves. For example, the processor (220) may determine that the third handwriting input is non-text using an artificial intelligence model stored in the memory (230). The artificial intelligence model may be implemented as a document layout analyzer. However, this is an example, and the processor (220) may also determine that the third handwriting input is non-text using a separate algorithm.

[0074] According to one embodiment, the processor (220) may obtain second coordinate information of at least one stroke included in the third handwriting input. For example, the processor (220) may obtain second coordinate information of at least one stroke included in the third handwriting input using a sensor.

[0075] According to one embodiment, the processor (220) may input second coordinate information into the generative AI model (360) in addition to the first coordinate information of the first handwriting input, the pen pressure information of the first handwriting input, and the pen speed information of the first handwriting input. According to one embodiment, the processor (220) may also input the third handwriting input into the generative AI model (360) instead of the second coordinate information.

[0076] According to one embodiment, the processor (220) may compare the first coordinate information of the first handwriting input with the second coordinate information of the third handwriting input using the generative AI model (360). According to one embodiment, the processor (220) may determine that the third handwriting input overlaps at least a portion of the first handwriting input based on the comparison result between the first coordinate information and the second coordinate information.

[0077] According to one embodiment, the processor (220) may obtain a fourth handwriting input (fourth handwriting) by modifying the third handwriting input such that the first plurality of strokes included in the first handwriting input and at least one stroke included in the third handwriting input are separated using the generative AI model (360). For example, the processor (220) may obtain the fourth handwriting input by adjusting the size or position of the third handwriting input.

[0078] According to one embodiment, the processor (220) can display the second handwriting input and the fourth handwriting input on the touch screen (260).

[0079] In one embodiment, the second handwriting input may represent handwriting obtained by transforming the first handwriting input using a generative AI model (360). In one embodiment, the fourth handwriting input may represent handwriting obtained by transforming the third handwriting input using a generative AI model (360).

[0080] FIG. 3 is a schematic block diagram of a memory according to one embodiment.

[0081] Referring to FIG. 3, the memory (230) may include a word analysis module (310), a text recognition module (320), a coordinate analysis module (330), a handwriting speed analysis module (340), a handwriting pressure analysis module (350), and a generative AI model (360). According to one embodiment, the word analysis module (310), the text recognition module (320), the coordinate analysis module (330), the handwriting speed analysis module (340), the handwriting pressure analysis module (350), and the generative AI model (360) may be implemented in software. Depending on the implementation, at least some of the word analysis module (310), the text recognition module (320), the coordinate analysis module (330), the handwriting speed analysis module (340), the handwriting pressure analysis module (350), and the generative AI model (360) may be implemented in hardware. According to one embodiment, the generative AI model (360) may include a text transformation module (361), a non-text transformation module (362), and a combination module (363).

[0082] According to one embodiment, a processor (220) (e.g., processor (220) of FIG. 2) can obtain a first handwriting input representing text and a third handwriting input representing non-text through a touch screen (260) (e.g., touch screen (260) of FIG. 2).

[0083] In one embodiment, the processor (220) may use the text recognition module (320) to determine that the first handwritten input is text and the third handwritten input is non-text. For example, the text recognition module (320) may include an artificial intelligence model implemented as a document layout analyzer.

[0084] In one embodiment, the processor (220) may distinguish the first handwritten input line by line using the word analysis module (310). In one embodiment, the processor (220) may distinguish the first handwritten input word by word using the word analysis module (310). For example, the word analysis module (310) may include an artificial intelligence model implemented as a document layout analyzer.

[0085] According to one embodiment, the processor (220) can obtain coordinate information of a first handwriting input and obtain coordinate information of a third handwriting input using a coordinate analysis module (330).

[0086] According to one embodiment, the processor (220) may obtain handwriting speed information of a first plurality of strokes included in a first handwriting input using a handwriting speed analysis module (340). According to one embodiment, the processor (220) may obtain handwriting speed information of the first handwriting input based on the distance between coordinates of points of the first plurality of strokes included in the first handwriting input using a handwriting speed analysis module (340).

[0087] According to one embodiment, the processor (220) may obtain pressure information of the first handwriting input using the pressure analysis module (350). For example, the processor (220) may obtain pressure information of the first handwriting input obtained by the stylus pen through the communication circuit (290) (e.g., the communication circuit (290) of FIG. 2) and store the pressure information in the pressure analysis module (350).

[0088] According to one embodiment, the processor (220) may input coordinate information, pen speed information, and pen pressure information of a first handwriting input representing text into the generative AI model (360). At this time, the processor (220) may further input information of the first handwriting input distinguished by word units and / or information of the first handwriting input distinguished by line units into the generative AI model (360).

[0089] According to one embodiment, the processor (220) may input coordinate information of a third handwriting input representing non-text into the generative AI model (360).

[0090] According to one embodiment, the text transformation module (361) can check whether the pressure identified at at least one point connecting the first stroke and the second stroke among the first plurality of strokes included in the first handwriting input is less than a specified pressure. The at least one point can represent a point included in a portion where the first stroke and the second stroke are connected to each other. According to one embodiment, the text transformation module (361) can obtain a second handwriting input that is obtained by transforming the first handwriting input so that the first stroke and the second stroke are separated, based on checking that the pressure identified at the at least one point is less than the specified pressure.

[0091] According to one embodiment, the text transformation module (361) may determine whether a pen speed identified between points connecting a first stroke and a second stroke among a first plurality of strokes included in a first handwriting input is greater than a specified speed. At least one point may represent a point included in a portion where the first stroke and the second stroke are connected to each other. According to one embodiment, the text transformation module (361) may obtain a second handwriting input that is a modified version of the first handwriting input such that the first stroke and the second stroke are separated, based on determining that a pen speed identified between points included in the first stroke or the second stroke is greater than a specified speed.

[0092] According to one embodiment, the non-text transformation module (362) may compare the coordinate information of the first handwriting input with the coordinate information of the third handwriting input. According to one embodiment, the non-text transformation module (362) may determine that the third handwriting input overlaps at least a portion of the first handwriting input based on the coordinate information of the first handwriting input and the coordinate information of the third handwriting input. According to one embodiment, the non-text transformation module (362) may obtain a fourth handwriting input by transforming the third handwriting input so that the third handwriting input is separated from the first handwriting input and the second handwriting input.

[0093] According to one embodiment, the combination module (363) can combine (combine) the second handwriting input and the fourth handwriting input.

[0094] The operations of the electronic device (201) described in the drawings below may be performed by the processor (220). However, for convenience of explanation, the operations performed by the processor (220) will be described as being performed by the electronic device (201).

[0095] FIG. 4 is a flowchart illustrating an operation of an electronic device according to one embodiment of the present invention to transform a handwriting input representing text.

[0096] Referring to FIG. 4, according to one embodiment, in operation 411, the electronic device (201) (e.g., the electronic device (201) of FIG. 2) may obtain a first handwriting input through the touch screen (260) (e.g., the touch screen (260) of FIG. 2). For example, the first handwriting input may be obtained by a stylus pen. According to one embodiment, the first handwriting input may include a plurality of strokes. According to one embodiment, a stroke may represent a continuous stroke from a point where an input is started to a point where the input is released. According to one embodiment, a stroke may include at least one point. According to one embodiment, the electronic device (201) may obtain a point at a specified time interval at a location where the stylus pen and the touch screen (260) are in contact.

[0097] According to one embodiment, in operation 413, the electronic device (201) can determine that the first handwriting input is text. For example, the electronic device (201) can determine whether the first handwriting input is text using an artificial intelligence model stored in the memory (230) (e.g., the memory (230) of FIG. 2 ). The artificial intelligence model can be implemented as a document layout analyzer. However, this is merely an example, and the electronic device (201) can also determine that the first handwriting input is text using a separate algorithm.

[0098] According to one embodiment, in operation 415, the electronic device (201) may obtain first coordinate information of points of a first plurality of strokes included in the first handwriting input based on determining that the first handwriting input is text.

[0099] According to one embodiment, in operation 417, the electronic device (201) may obtain pressure information on points of the first plurality of strokes based on determining that the first handwriting input is text. According to one embodiment, the electronic device (201) may receive pressure information on points of the first plurality of strokes sensed by a sensor included in the stylus pen through the communication circuit (290) (e.g., the communication circuit (290) of FIG. 2). According to one embodiment, the electronic device (201) may also obtain pressure information on points of the first plurality of strokes for the stylus pen using a sensor mounted on the touch screen (260).

[0100] According to one embodiment, in operation 419, the electronic device (201) may calculate (obtain, or compute) stroke speed information of the first plurality of strokes based on the distance between points of the first plurality of strokes, based on the determination that the first handwriting input is text. For example, the electronic device (201) may calculate (obtain, or compute) stroke speed information of a stroke connecting two adjacent points based on the distance between two adjacent points included in the first plurality of strokes.

[0101] According to one embodiment, in operation 421, the electronic device (201) may input first coordinate information, pen pressure information, and pen speed information into a generative AI model (360) (e.g., the generative AI model (360) of FIG. 3) to obtain a second handwriting input that is a modified version of the first handwriting input such that a first stroke corresponding to a first text included in a first plurality of strokes and a second stroke corresponding to a second text included in the first plurality of strokes are separated. For example, the electronic device (201) may delete a stroke corresponding to a portion where the first stroke and the second stroke are connected. For example, the electronic device (201) may also adjust the size of a stroke corresponding to a portion where the first stroke and the second stroke are connected to be smaller.

[0102] Additionally, according to one embodiment, the electronic device (201) can distinguish the first handwriting input on a line-by-line basis based on the first handwriting input being text. According to one embodiment, the electronic device (201) can obtain information about the first handwriting input distinguished on a line-by-line basis. According to one embodiment, the electronic device (201) can distinguish the first handwriting input on a word-by-word basis based on the first handwriting input being text. According to one embodiment, the electronic device (201) can obtain information about the first handwriting input distinguished on a word-by-word basis.

[0103] According to one embodiment, the electronic device (201) may obtain a second handwriting input by inputting information about the first handwriting input distinguished by line units and information about the first handwriting input distinguished by word units, in addition to the first coordinate information, the pen pressure information, and the pen speed information, into the generative AI model (360).

[0104] According to one embodiment, in operation 423, the electronic device (201) may display a second handwriting input on the touch screen (260).

[0105] Through this, the electronic device (201) according to one embodiment can provide a user with readable handwriting input by transforming overlapping handwriting inputs to be separated from each other.

[0106] FIG. 5A is a flowchart illustrating an operation of an electronic device according to one embodiment to transform handwriting input based on pressure.

[0107] Referring to FIG. 5A, according to one embodiment, in operation 511, the electronic device (201) (e.g., the electronic device (201) of FIG. 2) may input first coordinate information, pen pressure information, and pen speed information of the first handwriting input into the generative AI model (360) (e.g., the generative AI model (360) of FIG. 3). For example, the electronic device (201) may obtain pen pressure information sensed by a sensor included in a stylus pen through a communication circuit (290) (e.g., the communication circuit (290) of FIG. 2). For example, the electronic device (201) may also obtain pen pressure information by using a sensor included in a touch screen (260) (e.g., the touch screen (260) of FIG. 2).

[0108] According to one embodiment, in operation 513, the electronic device (201) may determine that the pressure detected at at least one point connecting the first stroke and the second stroke is less than a specified pressure. The at least one point may represent a point included in a portion where the first stroke and the second stroke are connected among the first plurality of strokes of the first handwriting input. According to one embodiment, if the generative AI model (360) determines that the pressure detected at at least one point connecting the first stroke and the second stroke is less than a specified pressure, the generative AI model (360) may modify the first handwriting input so that the first stroke and the second stroke are not connected.

[0109] According to one embodiment, in operation 515, the electronic device (201) may obtain a second handwriting input that is a modified version of the first handwriting input based on determining that the pressure detected at at least one point connecting the first stroke and the second stroke is less than a specified pressure. According to one embodiment, the second handwriting input may represent a separated handwriting input such that the first stroke and the second stroke are not connected to each other.

[0110] FIG. 5b is a flowchart illustrating an operation of an electronic device according to one embodiment to transform handwriting input based on a pen speed.

[0111] Referring to FIG. 5B, according to one embodiment, in operation 531, the electronic device (201) (e.g., the electronic device (201) of FIG. 2) may input first coordinate information, pen pressure information, and pen speed information into the generative AI model (360) (e.g., the generative AI model (360) of FIG. 3). For example, the electronic device (201) may obtain (calculate, or operate) pen speed information of the first plurality of strokes based on distances between at least some of the points of the first plurality of strokes included in the first handwriting input.

[0112] According to one embodiment, the electronic device (201) can check the coordinates of points of the first plurality of strokes at specified intervals. According to one embodiment, the electronic device (201) can calculate the stroke speed information of each of the first plurality of strokes based on the distance between the coordinates of two adjacent points of the first plurality of strokes.

[0113] For example, the electronic device (201) can obtain the coordinates of a first point among the plurality of points included in the first plurality of strokes at a first time. The electronic device (201) can obtain the coordinates of a second point among the plurality of points included in the first plurality of strokes, which is obtained after the first point, at a second time after a specified time from the first time. The electronic device (201) can calculate (obtain, or operate) stroke speed information connecting the first point and the second point based on the coordinates of the first point, the coordinates of the second point, and the specified time.

[0114] According to one embodiment, in operation 533, the electronic device (201) may determine that the pen speed determined based on the distance between points connecting the first stroke and the second stroke among the points included in the first plurality of strokes of the first handwriting input is greater than a specified speed. According to one embodiment, if the pen speed determined at the points connecting the first stroke and the second stroke is greater than the specified speed, the electronic device (201) may modify the first handwriting input so that the first stroke and the second stroke are not connected.

[0115] According to one embodiment, in operation 535, the electronic device (201) may obtain a second handwriting input that is a modified first handwriting input such that the first stroke and the second stroke are separated from each other based on determining that the determined handwriting speed is greater than a specified speed based on the distance between points connecting the first stroke and the second stroke.

[0116] FIG. 6 is a flowchart illustrating an operation of an electronic device according to one embodiment of the present invention to transform a handwriting input representing non-text.

[0117] Referring to FIG. 6, according to one embodiment, in operation 611, the electronic device (201) (e.g., the electronic device (201) of FIG. 2) may obtain a third handwriting input that at least partially overlaps the first handwriting input.

[0118] According to one embodiment, in operation 613, the electronic device (201) may determine that the third handwriting input is non-text. Non-text may include handwriting input other than text. For example, non-text may include a shape. For example, the electronic device (201) may determine that the third handwriting input is non-text using an artificial intelligence model stored in the memory (230) (e.g., the memory (230) of FIG. 2 ). The artificial intelligence model may be implemented as a document layout analyzer. However, this is merely an example, and the electronic device (201) may also determine that the third handwriting input is non-text using a separate algorithm.

[0119] According to one embodiment, in operation 615, the electronic device (201) may obtain second coordinate information of at least one stroke included in the third handwriting input.

[0120] According to one embodiment, in operation 617, the electronic device (201) may further input second coordinate information into the generative AI model (360) (e.g., the generative AI model (360) of FIG. 3) to obtain a fourth handwriting input that is transformed from the third handwriting input such that at least one stroke is separated from the first plurality of strokes included in the first handwriting input. According to one embodiment, the electronic device (201) may use the generative AI model (360) to compare first coordinate information of the first plurality of strokes included in the first handwriting input with second coordinate information of at least one stroke included in the third handwriting input. According to one embodiment, the electronic device (201) may determine that the third handwriting input overlaps at least a portion of the first handwriting input based on the first coordinate information and the second coordinate information. For example, the electronic device (201) may obtain the fourth handwriting input by adjusting the size or position of the third handwriting input such that at least one stroke is separated from the first plurality of strokes included in the first handwriting input.

[0121] According to one embodiment, in operation 619, the electronic device (201) may display a second handwriting input that is a modified version of the first handwriting input and a fourth handwriting input that is a modified version of the third handwriting input on the touch screen (260).

[0122] FIG. 7 is a diagram illustrating an operation of an electronic device according to one embodiment to distinguish a first handwriting input by word and line.

[0123] Referring to FIG. 7, according to one embodiment, an electronic device (201) (e.g., the electronic device (201) of FIG. 2) may obtain a first handwriting input (700) (e.g., “Today is Sunday, hello”) through a touch screen (260) (e.g., the touch screen (260) of FIG. 2). According to one embodiment, the first handwriting input (700) may represent a plurality of texts.

[0124] According to one embodiment, the electronic device (201) can distinguish the first handwriting input (700) on a line-by-line basis. For example, the electronic device (201) can distinguish the first handwriting input (700) into handwriting inputs (710, 720) located on a first line and handwriting inputs (730) located on a second line.

[0125] According to one embodiment, the electronic device (201) can distinguish the first handwriting input (700) on a word-by-word basis. According to one embodiment, the electronic device (201) can distinguish the first handwriting input (700) on a word-by-word basis based on spacing (or spacing spaces). For example, the electronic device (201) can distinguish the first handwriting input (700) into a handwriting input (710) representing a first word, a handwriting input (720) representing a second word, and a handwriting input (730) representing a third word.

[0126] According to one embodiment, the electronic device (201) can obtain (calculate or operate) coordinate information, pen speed information, and pen pressure information of each handwriting input distinguished by line and word units.

[0127] FIG. 8 is a drawing illustrating a portion of a first handwriting input to explain an operation of an electronic device according to one embodiment of the present invention to obtain handwriting speed information of a first handwriting input.

[0128] Referring to FIG. 8, according to one embodiment, the first plurality of strokes of the first handwriting input (700 of FIG. 7) may include a plurality of points. According to one embodiment, the electronic device (201) (e.g., the electronic device (201) of FIG. 2) may obtain coordinate information of each of the plurality of points. According to one embodiment, the electronic device (201) may obtain (calculate, or operate) handwriting speed information of the first handwriting input based on the distance between the coordinates of the plurality of points.

[0129] According to one embodiment, the electronic device (201) may determine a portion (810) of a stroke among a first plurality of strokes having stroke speed information greater than a specified speed. For example, according to one embodiment, the electronic device (201) may obtain a first point (811) at a first time. The electronic device (201) may obtain a second point (812) after the first point (811) at a second time, which is a specified time after the first time. According to one embodiment, the electronic device (201) may obtain (calculate, or operate) stroke speed information of a stroke connecting the first point (811) and the second point (812) based on the specified time, the coordinates corresponding to the first point (811), and the coordinates corresponding to the second point (812). The electronic device (201) may determine that the stroke speed information of the stroke connecting the first point (811) and the second point (812) is greater than the specified speed.

[0130] According to one embodiment, the electronic device (201) may determine a portion (820) of a stroke having a stroke speed information that is not greater than a specified speed among the first plurality of strokes. For example, according to one embodiment, the electronic device (201) may obtain a third point (821) at a third time. The electronic device (201) may obtain a fourth point (822) after the third point (821) at a fourth time, which is a specified time after the third time. The electronic device (201) may obtain (calculate, or operate) stroke speed information of a stroke connecting the third point (821) and the fourth point (822) based on the specified time, the coordinates corresponding to the third point (821), and the coordinates corresponding to the fourth point (822).

[0131] FIG. 9 is a drawing for explaining a portion in which a plurality of strokes included in a first handwriting input are connected by an electronic device according to one embodiment.

[0132] Referring to FIG. 9, according to one embodiment, an electronic device (201) (e.g., the electronic device (201) of FIG. 2) can identify portions (910, 920, 930, 940, 950, 960, 970, 980, 990, 991) where at least two strokes among a plurality of strokes are connected to each other based on at least one of the pen speed information or the pen pressure information of the plurality of strokes included in the first handwriting input (700 of FIG. 7) using a generative AI model (360) (e.g., the generative AI model (360) of FIG. 3).

[0133] According to one embodiment, the pressure detected in each of the connected portions of the strokes (910, 920, 930, 940, 950, 960, 970, 980, 990, 991) may be lower than the specified pressure.

[0134] According to one embodiment, the stroke speed identified in each of the connected portions (910, 920, 930, 940, 950, 960, 970, 980, 990, 991) may be greater than the specified speed.

[0135] FIG. 10 is a diagram for explaining an operation of an electronic device according to one embodiment of the present invention to obtain a second handwriting input that is a modified first handwriting input.

[0136] Referring to FIG. 10, according to one embodiment, an electronic device (201) (e.g., the electronic device (201) of FIG. 2) may obtain a second handwriting input (1000) that is a modified first handwriting input (700 of FIG. 7) such that a plurality of strokes included in the first handwriting input are separated from each other, by using a generative AI model (360) (e.g., the generative AI model (360) of FIG. 3).

[0137] For example, the electronic device (201) can change the shape of portions (910, 920, 930, 940, 950, 960, 970, 980, 990, 991 of FIG. 9) where at least two strokes among the plurality of strokes are connected to each other. For example, the electronic device (201) can delete portions (910, 920, 930, 940, 950, 960, 970, 980, 990, 991 of FIG. 9) where at least two strokes among the plurality of strokes are connected to each other.

[0138] FIG. 11 is a diagram illustrating an electronic device according to one embodiment of the present invention, wherein the electronic device illustrates a first handwriting input and a third handwriting input overlapping at least a portion of the first handwriting input. FIG. 12 is a diagram illustrating an operation of the electronic device according to one embodiment of the present invention of obtaining a fourth handwriting input that is a modified version of the third handwriting input.

[0139] Referring to FIG. 11, an electronic device (201) (e.g., the electronic device (201) of FIG. 2) can obtain third handwriting inputs (1110, 1120, 1130) through a touch screen (260) (e.g., the touch screen (260) of FIG. 2). For example, the third handwriting inputs (1110, 1120, 1130) can represent non-text. In one embodiment, non-text can include handwriting inputs other than text. For example, non-text can include shapes.

[0140] According to one embodiment, the electronic device (201) can obtain coordinate information of the third handwriting input (1110, 1120, 1130).

[0141] According to one embodiment, the electronic device (201) can input coordinate information of the third handwriting input (1110, 1120, 1130) into a generative AI model (360) (e.g., the generative AI model (360) of FIG. 3) stored in a memory (230) (e.g., the memory (230) of FIG. 2).

[0142] According to one embodiment, the electronic device (201) can use the generative AI model (360) to determine that the third handwriting input (1110, 1120, 1130) overlaps at least a portion of the first handwriting input (700 of FIG. 7).

[0143] Referring to FIG. 12, the electronic device (201) may obtain a second handwriting input (1000 of FIG. 10) that is a modified version of the first handwriting input (700) and a fourth handwriting input (1210, 1220, 1230) that is a modified version of the third handwriting input (1110, 1120, 1130) based on inputting coordinate information of the third handwriting input (1110, 1120, 1130) into the generative AI model (360). According to one embodiment, the fourth handwriting input (1210, 1220, 1230) may not overlap with the first handwriting input (700) and / or the second handwriting input (1000).

[0144] For example, the electronic device (201) can obtain a fourth handwriting input (1210, 1220, 1230) that is a modified version of the third handwriting input (1110, 1120, 1130) by adjusting the position, size, or shape of the third handwriting input (1110, 1120, 1130).

[0145] According to one embodiment, an electronic device (201) (e.g., electronic device (201) of FIG. 2) may include a touch screen (260) (e.g., touch screen (260) of FIG. 2), at least one processor (220) (e.g., processor (220) of FIG. 2), and a memory (230) (e.g., memory (230) of FIG. 2) for storing instructions.

[0146] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to obtain a first handwriting through the touch screen (260).

[0147] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to obtain first coordinate information of points of a first plurality of strokes included in the first handwriting.

[0148] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to obtain pressure information identified at the points of the first plurality of strokes.

[0149] According to one embodiment, the instructions, when executed by the at least one processor (220), cause the electronic device (201) to provide the first coordinate information, the pen pressure information, and the pen speed information to a generative AI model (360) (e.g., the generative AI model (360) of FIG. 3) to obtain a second handwriting transformed from the first handwriting, wherein a first stroke and a second stroke that are connected to each other in the first handwriting are separated from the second handwriting.

[0150] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to display the second handwriting on the touch screen (260).

[0151] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to obtain stroke speed information of the first plurality of strokes based on distances between at least some of the points of the first plurality of strokes, and to provide the stroke speed information to the generative AI model.

[0152] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to acquire the second stroke if the pressure identified in the portion where the second stroke overlaps at least a portion of the first stroke is less than a specified pressure.

[0153] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to acquire the second stroke if the identified stroke speed in the portion where the second stroke overlaps at least a portion of the first stroke is greater than a specified speed.

[0154] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to obtain, through the communication circuit (290), the pressure information of the points of the first plurality of strokes sensed by a sensor included in the stylus pen and transmitted by the stylus pen.

[0155] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to check coordinates of the points of the first plurality of strokes at specified intervals and to obtain the stroke speed information of the first plurality of strokes based on a distance between at least some of the coordinates.

[0156] According to one embodiment, the instructions, when executed by the at least one processor (220), cause the electronic device (201) to obtain a third handwriting that is a non-text that at least partially overlaps the first handwriting through the touch screen (260), obtain second coordinate information of at least one stroke included in the third handwriting, and provide the second coordinate information to the generative AI model to obtain a fourth handwriting that is a modified version of the third handwriting, wherein the at least one stroke included in the third handwriting is separated from the first plurality of strokes included in the first handwriting, and display the second handwriting and the fourth handwriting on the touch screen (260).

[0157] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to provide the first coordinate information, the pen pressure information, and the pen speed information to the generative AI model (360) based on determining that the first handwriting is text, and to provide the second coordinate information to the generative AI model (360) based on determining that the third handwriting is non-text.

[0158] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to distinguish the first handwriting into word units and provide information about the first handwriting distinguished into word units to the generative AI model (360).

[0159] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to distinguish the first handwriting into line units and provide information about the first handwriting distinguished into line units to the generative AI model (360).

[0160] According to one embodiment, the instructions, when executed by the at least one processor (220), may cause the electronic device (201) to provide the first handwriting to the generative AI model (360) to obtain the second handwriting.

[0161] According to one embodiment, a method of operating an electronic device (201) may include an operation of obtaining a first handwriting through a touch screen (260) included in the electronic device (201).

[0162] According to one embodiment, a method of operating an electronic device (201) may include an operation of obtaining first coordinate information of points of a first plurality of strokes included in the first handwriting.

[0163] According to one embodiment, the method of operating the electronic device (201) may include an operation of obtaining pressure information identified at the points of the first plurality of strokes.

[0164] According to one embodiment, the method of operating the electronic device (201) includes providing the first coordinate information, the pen pressure information, and the pen speed information to a generative AI model (360) to obtain a second handwriting transformed from the first handwriting, wherein the first stroke and the second stroke connected to each other in the first handwriting can be separated from the second handwriting.

[0165] According to one embodiment, the method of operating the electronic device (201) may include an operation of displaying the second handwriting on the touch screen (260).

[0166] According to one embodiment, the method of operating the electronic device (201) may include an operation of obtaining stroke speed information of the first plurality of strokes based on a distance between at least some of the points of the first plurality of strokes, and an operation of providing the stroke speed information to the generative AI model (360).

[0167] According to one embodiment, the method of operating the electronic device (201) may include an operation of acquiring the second writing when the pressure identified in the portion where the second stroke overlaps at least a portion of the first stroke is less than a specified pressure.

[0168] According to one embodiment, the method of operating the electronic device (201) may include an operation of acquiring the second writing when the pen speed identified in the portion where the second stroke overlaps at least a portion of the first stroke is greater than a specified speed.

[0169] According to one embodiment, a method of operating an electronic device (201) may include an operation of obtaining pressure information of the points of the first plurality of strokes sensed by a sensor included in a stylus pen and transmitted by the stylus pen through a communication circuit included in the electronic device.

[0170] According to one embodiment, the operating method of the electronic device (201) may include an operation of checking coordinates of the points of the first plurality of strokes at specified intervals and an operation of obtaining the pen speed information of the first plurality of strokes based on a distance between at least some of the coordinates.

[0171] According to one embodiment, the operating method of the electronic device (201) may include an operation of obtaining a third handwriting that is a non-text that overlaps at least a portion of the first handwriting through the touch screen (260), an operation of obtaining second coordinate information of at least one stroke included in the third handwriting, an operation of providing the second coordinate information to the generative AI model (360) to obtain a fourth handwriting that is a modified version of the third handwriting, wherein the at least one stroke included in the third handwriting is separated from the first plurality of strokes included in the first handwriting, and an operation of displaying the second handwriting and the fourth handwriting on the touch screen (260).

[0172] According to one embodiment, the operating method of the electronic device (201) may include an operation of providing the first coordinate information, the pen pressure information, and the pen speed information to the generative AI model (360) based on determining that the first handwriting is text, and an operation of providing the second coordinate information to the generative AI model (360) based on determining that the third handwriting is non-text.

[0173] According to one embodiment, the operating method of the electronic device (201) may include an operation of distinguishing the first handwriting into word units and an operation of providing information about the first handwriting distinguished into word units to the generative AI model (360).

[0174] According to one embodiment, the operating method of the electronic device (201) may include an operation of distinguishing the first handwriting in units of lines and an operation of providing information about the first handwriting distinguished in units of lines to the generative AI model (360).

[0175] According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor (220) of an electronic device (201), cause the electronic device (201) to perform at least one operation, wherein the at least one operation may include an operation of obtaining a first handwriting through a touch screen (260) included in the electronic device (201).

[0176] According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor (220) of an electronic device (201), cause the electronic device (201) to perform at least one operation, wherein the at least one operation may include an operation of obtaining first coordinate information of points of a first plurality of strokes included in the first handwriting.

[0177] According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor (220) of an electronic device (201), cause the electronic device (201) to perform at least one operation, wherein the at least one operation may include an operation of obtaining pressure information for the points of the first plurality of strokes.

[0178] According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor (220) of an electronic device (201), cause the electronic device (201) to perform at least one operation, the at least one operation including providing the first coordinate information and the pen pressure information to a generative AI model (360) to obtain a second pen that is transformed from the first pen, wherein a first stroke and a second stroke that are connected to each other in the first pen can be separated from the second pen.

[0179] According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor (220) of an electronic device (201), cause the electronic device (201) to perform at least one operation, wherein the at least one operation may include an operation of displaying the second handwriting on the touch screen (260).

[0180] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments disclosed in this document are not limited to the aforementioned devices.

[0181] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0182] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0183] Various embodiments of the present document may be implemented as software (e.g., program (140)) including one or more commands stored in a storage medium (e.g., built-in memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101, 201)). For example, a processor (e.g., processor (120, 220)) of a machine (e.g., electronic device (101, 201)) may call at least one command among the one or more commands stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the called at least one command. The one or more commands may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means a device in which the storage medium is tangible, It simply means that it does not contain signals (e.g. electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on a storage medium.

[0184] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0185] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (201), Touch screen (260); at least one processor (220); and Contains a memory (230) for storing instructions, The above instructions, when executed by the at least one processor, cause the electronic device to: Obtain the first handwriting through the above touch screen, Obtain first coordinate information of points of the first plurality of strokes included in the first writing, Obtain pressure information for the points of the first plurality of strokes, The first coordinate information and the pressure information are provided to a generative AI model (360) to obtain a second handwriting transformed from the first handwriting, wherein the first stroke and the second stroke connected to each other in the first handwriting are separated from the second handwriting, An electronic device that displays the second handwriting on the touch screen.

2. In paragraph 1, The above instructions, when executed by the at least one processor, cause the electronic device to: Obtaining stroke speed information of the first plurality of strokes based on the distance between at least some of the points of the first plurality of strokes, An electronic device that provides the above-mentioned writing information to the above-mentioned generative AI model (360).

3. In any one of paragraphs 1 and 2, The above instructions, when executed by the at least one processor, cause the electronic device to: An electronic device that acquires the second writing when the pressure detected at a portion where the second stroke overlaps at least a portion of the first stroke is less than a specified pressure.

4. In any one of paragraphs 1 to 3, The above instructions, when executed by the at least one processor, cause the electronic device to: An electronic device that acquires the second stroke when the speed identified in the portion where the second stroke overlaps at least a portion of the first stroke is greater than a specified speed.

5. In any one of paragraphs 1 to 4, Further comprising a communication circuit, The above instructions, when executed by the at least one processor, cause the electronic device to: An electronic device that obtains pressure information of the points of the first plurality of strokes sensed by a sensor included in the stylus pen and transmitted by the stylus pen through the communication circuit.

6. In any one of paragraphs 1 to 5, The above instructions, when executed by the at least one processor, cause the electronic device to: Check the coordinates of the points of the first plurality of strokes at specified intervals, An electronic device for obtaining the stroke speed information of the first plurality of strokes based on a distance between at least some of the coordinates.

7. In any one of paragraphs 1 to 6, The above instructions, when executed by the at least one processor, cause the electronic device to: Obtaining a third non-text handwriting that at least partially overlaps the first handwriting through the touch screen, Obtain second coordinate information of at least one stroke included in the third writing, The second coordinate information is provided to the generative AI model to obtain a fourth handwriting that is a modified version of the third handwriting, wherein at least one stroke included in the third handwriting is separated from the first plurality of strokes included in the first handwriting, An electronic device that displays the second handwriting and the fourth handwriting on the touch screen.

8. In any one of paragraphs 1 to 7, The above instructions, when executed by the at least one processor, cause the electronic device to: Based on the confirmation that the first handwriting is text, the first coordinate information, the pen pressure information, and the pen speed information are provided to the generative AI model, An electronic device that provides the second coordinate information to the generative AI model based on determining that the third handwriting is non-text.

9. In any one of paragraphs 1 to 8, The above instructions, when executed by the at least one processor, cause the electronic device to: Distinguish the above first writing into word units, An electronic device that provides information about the first handwriting distinguished by the above word units to the generative AI model.

10. In any one of paragraphs 1 to 9, The above instructions, when executed by the at least one processor, cause the electronic device to: The above first writing is distinguished by line units, An electronic device that provides information about the first handwriting distinguished by the above line units to the generative AI model.

11. In any one of paragraphs 1 to 10, The above instructions, when executed by the at least one processor, cause the electronic device to: An electronic device that provides the first handwriting to the generative AI model stored in the memory to obtain the second handwriting.

12. In the operating method of an electronic device (201), An action of obtaining a first handwriting through a touch screen (260) included in the electronic device; An operation of obtaining first coordinate information of points of a first plurality of strokes included in the first writing; An operation of obtaining pressure information for the points of the first plurality of strokes; An operation of providing the first coordinate information and the pressure information to a generative AI model (360) to obtain a second handwriting transformed from the first handwriting, wherein the first stroke and the second stroke connected to each other in the first handwriting are separated from the second handwriting; and A method of operating an electronic device including an action of displaying the second handwriting on the touch screen.

13. In paragraph 12, A method of operating an electronic device further comprising at least one operation of the electronic device according to any one of claims 2 to 11.

14. In a non-transitory computer-readable storage medium storing instructions, the instructions, when executed by at least one processor (220) of an electronic device, cause the electronic device (201) to perform operations, the operations comprising: An action of obtaining a first handwriting through a touch screen (260) included in the electronic device; An operation of obtaining first coordinate information of points of a first plurality of strokes included in the first writing; An operation of obtaining pressure information identified at the points of the first plurality of strokes; An operation of providing the first coordinate information and the pressure information to a generative AI model (360) to obtain a second handwriting transformed from the first handwriting, wherein the first stroke and the second stroke connected to each other in the first handwriting are separated from the second handwriting; and A storage medium including an action of displaying the second handwriting on the touch screen.

15. In paragraph 14, A storage medium further comprising at least one operation of an electronic device according to any one of claims 2 to 11.

Images